{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# snnTorch - Tutorial 4\n",
    "### By Jason K. Eshraghian"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Note: explanation is yet to be added. For now, only code is updated to show how to automatically implement TBPTT.\n",
    "\n",
    "# Truncated Backpropagation through time\n",
    "In this tutorial, we'll use a convolutional neural network (CNN) to classify the MNIST dataset.\n",
    "We will use the truncated backpropagation through time (BPTT) algorithm to do so. This tutorial is largely the same as tutorial 2, just with a different network architecture to show how to integrate convolutions with snnTorch.\n",
    "\n",
    "If running in Google Colab:\n",
    "* Ensure you are connected to GPU by checking Runtime > Change runtime type > Hardware accelerator: GPU\n",
    "* Next, install the Test PyPi distribution of snnTorch by clicking into the following cell and pressing `Shift+Enter`."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "# Install the test PyPi Distribution of snntorch\n",
    "!pip install -i https://test.pypi.org/simple/ snntorch"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Looking in indexes: https://test.pypi.org/simple/\n",
      "Requirement already satisfied: snntorch in c:\\users\\jason\\dropbox\\repos\\snntorch (0.0.7)\n"
     ]
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 1. Setting up the Static MNIST Dataset\n",
    "### 1.1. Import packages and setup environment"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "import snntorch as snn\n",
    "from snntorch import backprop as bp\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import DataLoader\n",
    "from torchvision import datasets, transforms\n",
    "import numpy as np\n",
    "import itertools\n",
    "import matplotlib.pyplot as plt"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 1.2 Define network and SNN parameters\n",
    "We will use a 2conv-2MaxPool-FCN architecture for a sequence of 25 time steps.\n",
    "\n",
    "* `alpha` is the decay rate of the synaptic current of a neuron\n",
    "* `beta` is the decay rate of the membrane potential of a neuron"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "# Network Architecture\n",
    "num_inputs = 28*28\n",
    "num_hidden = 1000\n",
    "num_outputs = 10\n",
    "\n",
    "# Training Parameters\n",
    "batch_size=128\n",
    "data_path='/data/mnist'\n",
    "\n",
    "# Temporal Dynamics\n",
    "num_steps = 25\n",
    "time_step = 1e-3\n",
    "tau_mem = 4e-3\n",
    "tau_syn = 3e-3\n",
    "alpha = float(np.exp(-time_step/tau_syn))\n",
    "beta = float(np.exp(-time_step/tau_mem))\n",
    "\n",
    "dtype = torch.float\n",
    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 1.3 Download MNIST Dataset\n",
    "To see how to construct a validation set, refer to Tutorial 1."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "# Define a transform\n",
    "transform = transforms.Compose([\n",
    "            transforms.Resize((28, 28)),\n",
    "            transforms.Grayscale(),\n",
    "            transforms.ToTensor(),\n",
    "            transforms.Normalize((0,), (1,))])\n",
    "\n",
    "mnist_train = datasets.MNIST(data_path, train=True, download=True, transform=transform)\n",
    "mnist_test = datasets.MNIST(data_path, train=False, download=True, transform=transform)"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 1.4 Create DataLoaders"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "train_loader = DataLoader(mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\r\n",
    "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=True)"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 2. Define Network\n",
    "snnTorch contains a series of neuron models and related functions to ease the training process.\n",
    "Neurons are treated as activations with recurrent connections, and integrate smoothly with PyTorch's pre-existing layer functions.\n",
    "* `snntorch.Stein` is a simple Leaky Integrate and Fire (LIF) neuron. Specifically, it uses Stein's model which assumes instantaneous rise times for synaptic current and membrane potential.\n",
    "* `snntorch.FastSigmoidSurrogate` defines separate forward and backward functions. The forward function is a Heaviside step function for spike generation. The backward function is the derivative of a fast sigmoid function, to ensure continuous differentiability.\n",
    "FSS is mostly derived from:\n",
    "\n",
    ">Neftci, E. O., Mostafa, H., and Zenke, F. (2019) Surrogate Gradient Learning in Spiking Neural Networks. https://arxiv.org/abs/1901/09948\n",
    "\n",
    "There are a few other surrogate gradient functions included.\n",
    "`snn.slope` is a variable that defines the slope of the backward surrogate.\n",
    "TO-DO: Include visualisation.\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we can define our spiking neural network (SNN).\r\n",
    "If you have already worked through Tutorial 2, you may wish to skip ahead.\r\n",
    "\r\n",
    "The init_hidden argument will initialize the hidden states & spike outputs as instance variables.\r\n",
    "Although the forward method looks messier with the calls to these instance variables, it eliminates the need to detach all the variables from the computational graph manually."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "class Net(nn.Module):\r\n",
    "    def __init__(self):\r\n",
    "        super().__init__()\r\n",
    "\r\n",
    "    # initialize layers\r\n",
    "        snn.LIF.clear_instances() # boilerplate\r\n",
    "        self.fc1 = nn.Linear(num_inputs, num_hidden)\r\n",
    "        self.lif1 = snn.Stein(alpha=alpha, beta=beta, num_inputs=num_hidden, batch_size=batch_size, init_hidden=True)\r\n",
    "        self.fc2 = nn.Linear(num_hidden, num_outputs)\r\n",
    "        self.lif2 = snn.Stein(alpha=alpha, beta=beta, num_inputs=num_outputs, batch_size=batch_size, init_hidden=True)\r\n",
    "\r\n",
    "\r\n",
    "    def forward(self, x):\r\n",
    "        cur1 = self.fc1(x)\r\n",
    "        self.lif1.spk1, self.lif1.syn1, self.lif1.mem1 = self.lif1(cur1, self.lif1.syn, self.lif1.mem)\r\n",
    "        cur2 = self.fc2(self.lif1.spk)\r\n",
    "        self.lif2.spk, self.lif2.syn, self.lif2.mem = self.lif2(cur2, self.lif2.syn, self.lif2.mem)\r\n",
    "\r\n",
    "        return self.lif2.spk, self.lif2.mem\r\n",
    "\r\n",
    "net = Net().to(device)"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3. Training\n",
    "Time for training! Let's define our train and test functions."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "def train(net, device, train_loader, optimizer, criterion, epoch):\r\n",
    "    for batch_idx, (data, target) in enumerate(train_loader):\r\n",
    "        data, target = data.to(device), target.to(device)\r\n",
    "\r\n",
    "        loss = bp.TBPTT(net, data, target, num_steps, batch_size, optimizer, criterion, K=K)\r\n",
    "        # loss = bp.BPTT(net, data, target, num_steps, batch_size, optimizer, criterion)\r\n",
    "\r\n",
    "        if batch_idx % 20 == 0:\r\n",
    "            print(f\"Train Epoch: {epoch} [{batch_idx*len(data)}/{len(train_loader.dataset)}], \"\r\n",
    "                  f\"Loss: {loss.item()}\")\r\n",
    "    loss_hist.append(loss.item())  # only recording at the end of each epoch\r\n",
    "\r\n",
    "\r\n",
    "def test(net, device, test_loader, criterion):\r\n",
    "    net.eval()\r\n",
    "    test_loss = 0\r\n",
    "    correct = 0\r\n",
    "    with torch.no_grad():\r\n",
    "        for data, target in test_loader:\r\n",
    "            data, target = data.to(device), target.to(device)\r\n",
    "\r\n",
    "            spk2_rec = []\r\n",
    "            snn.Stein.zeros_hidden()  # reset hidden states to 0\r\n",
    "            if data.size()[0] == batch_size:\r\n",
    "                for step in range(num_steps):\r\n",
    "                    spk2, mem2 = net(data.view(batch_size, -1))\r\n",
    "                    spk2_rec.append(spk2)\r\n",
    "\r\n",
    "                # Test Loss where batch=128; only calc on final time step\r\n",
    "                # log_p_ytest = log_softmax_fn(mem2)\r\n",
    "                test_loss += criterion(mem2, target)\r\n",
    "                # Test Acc where batch=128\r\n",
    "                _, idx = torch.stack(spk2_rec, dim=0).sum(dim=0).max(1)  # predicted indexes\r\n",
    "                correct += sum((target == idx).cpu().numpy())\r\n",
    "                # print(correct)\r\n",
    "\r\n",
    "            else:  # Handle drop_last = False\r\n",
    "                temp_data = torch.zeros((batch_size, *(data[0].size())), dtype=dtype, device=device)  # pad out temp_data now\r\n",
    "                temp_data[:(data.size()[0])] = data\r\n",
    "\r\n",
    "                for step in range(num_steps):\r\n",
    "                    spk2, mem2 = net(temp_data.view(batch_size, -1))\r\n",
    "                    spk2_rec.append(spk2)\r\n",
    "\r\n",
    "                # Test set loss - only calc on the final time-step\r\n",
    "                # log_p_ytest = log_softmax_fn(mem2[:data.size()[0]])\r\n",
    "                test_loss += criterion(mem2[:data.size()[0]], target)\r\n",
    "                # Test Acc where batch=128\r\n",
    "                _, idx = torch.stack(spk2_rec, dim=0).sum(dim=0).max(1)  # predicted indexes\r\n",
    "                correct += sum((target == idx[:data.size()[0]]).cpu().numpy())\r\n",
    "\r\n",
    "        test_loss_hist.append(test_loss.item())\r\n",
    "        test_acc = correct / len(test_loader.dataset)\r\n",
    "        print(f\"\\nTest set: Average loss: {(test_loss/(len(test_loader.dataset)/batch_size))}, Accuracy: [{correct}/{len(test_loader.dataset)}] ({(correct/len(test_loader.dataset))})\\n\"\r\n",
    "              f\"=====================\\n\")\r\n",
    "\r\n",
    "        return test_loss, test_acc, spk2_rec"
   ],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.1 Optimizer & Loss\n",
    "* *Output Activation*: We'll apply the softmax function to the membrane potentials of the output layer, rather than the spikes.\n",
    "* *Loss*: This will then be used to calculate the negative log-likelihood loss.\n",
    "By encouraging the membrane of the correct neuron class to reach the threshold, we expect that neuron will fire more frequently.\n",
    "The loss could be applied to the spike count as well, but the membrane is  continuous whereas spike count is discrete.\n",
    "* *Optimizer*: The Adam optimizer is used for weight updates.\n",
    "* *Accuracy*: Accuracy is measured by counting the spikes of the output neurons. The neuron that fires the most frequently will be our predicted class.\n"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.2 Training Loop\n",
    "Now just sit back, relax, and wait for convergence."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "no_trials = 1\r\n",
    "# lr_values = [1e-3, 5e-4, 1e-4] # these values are good\r\n",
    "lr_values = [1e-4]\r\n",
    "batch_size = 128\r\n",
    "data_path = '/data/mnist'\r\n",
    "# subset = 50  # can remove this line in Colab\r\n",
    "num_steps = 25\r\n",
    "epochs = 1\r\n",
    "betas = (0.9, 0.999)\r\n",
    "K = 25  # number of time steps to accumulate over -- right now I'm using BPTT anyway so this is ignored\r\n",
    "SAVE_GOOGLE_COLAB = False\r\n",
    "dtype = torch.float\r\n",
    "device = torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\")\r\n",
    "transform = transforms.Compose([\r\n",
    "    transforms.Resize((28, 28)),\r\n",
    "    transforms.Grayscale(),\r\n",
    "    transforms.ToTensor(),\r\n",
    "    transforms.Normalize((0,), (1,))\r\n",
    "])\r\n",
    "mnist_train = datasets.MNIST(data_path, train=True, download=True, transform=transform)\r\n",
    "mnist_test = datasets.MNIST(data_path, train=False, download=True, transform=transform)\r\n",
    "# mnist_train = data_subset(mnist_train, subset)  # reduce dataset by x100 - can remove this line in Colab\r\n",
    "# mnist_test = data_subset(mnist_test, subset)\r\n",
    "train_loader = DataLoader(mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\r\n",
    "test_loader = DataLoader(mnist_test, batch_size=batch_size, shuffle=True, drop_last=False)\r\n",
    "\r\n",
    "# Adam\r\n",
    "# df = pd.DataFrame(columns=['lr', 'epoch', 'test_set_loss', 'test_set_accuracy'])\r\n",
    "for i in range(no_trials):\r\n",
    "    for lr in lr_values:\r\n",
    "        net = Net().to(device)\r\n",
    "        optimizer = torch.optim.Adam(net.parameters(), lr=lr, betas=betas)\r\n",
    "        # log_softmax_fn = nn.LogSoftmax(dim=-1)\r\n",
    "        criterion = nn.CrossEntropyLoss() # note: CrossEntropy dims must be B x num_classes. Can increase dimensionality, read docs.\r\n",
    "\r\n",
    "        loss_hist = []\r\n",
    "        test_loss_hist = []\r\n",
    "        print(f\"========Trial: {i}, Learning Rate: {lr}\")\r\n",
    "        for epoch in range(epochs):\r\n",
    "            train(net, device, train_loader, optimizer, criterion, epoch)\r\n",
    "            test_loss, test_acc, _ = test(net, device, test_loader, criterion)\r\n",
    "\r\n",
    "            # df = df.append(\r\n",
    "            #     {'trial': i, 'lr': lr, 'epoch': epoch, 'test_set_loss': test_loss.item(),\r\n",
    "            #      'test_set_accuracy': test_acc}, ignore_index=True)\r\n",
    "            # df.to_csv('Adam_BPTT2.csv', index=False)\r\n",
    "            # if SAVE_GOOGLE_COLAB:\r\n",
    "            #     shutil.copy(\"Adam_BPTT.csv\", \"/content/Adam_BPTT.csv\")\r\n",
    "\r\n",
    "\r\n",
    "loss_hist_true_grad = loss_hist\r\n",
    "test_loss_hist_true_grad = test_loss_hist"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "========Trial: 0, Learning Rate: 0.0001\n",
      "Train Epoch: 0 [0/60000], Loss: 87.99430084228516\n",
      "Train Epoch: 0 [2560/60000], Loss: 31.979496002197266\n",
      "Train Epoch: 0 [5120/60000], Loss: 20.729297637939453\n",
      "Train Epoch: 0 [7680/60000], Loss: 21.22612762451172\n",
      "Train Epoch: 0 [10240/60000], Loss: 17.037744522094727\n",
      "Train Epoch: 0 [12800/60000], Loss: 16.702619552612305\n",
      "Train Epoch: 0 [15360/60000], Loss: 15.043231010437012\n",
      "Train Epoch: 0 [17920/60000], Loss: 15.852062225341797\n",
      "Train Epoch: 0 [20480/60000], Loss: 18.419878005981445\n",
      "Train Epoch: 0 [23040/60000], Loss: 15.934926986694336\n",
      "Train Epoch: 0 [25600/60000], Loss: 14.167062759399414\n",
      "Train Epoch: 0 [28160/60000], Loss: 13.726014137268066\n"
     ]
    },
    {
     "output_type": "error",
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-9-8c324fbb3d18>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m     38\u001b[0m         \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34mf\"========Trial: {i}, Learning Rate: {lr}\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     39\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0mepoch\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mepochs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 40\u001b[1;33m             \u001b[0mtrain\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdevice\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtrain_loader\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moptimizer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mepoch\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     41\u001b[0m             \u001b[0mtest_loss\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtest_acc\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0m_\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtest\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdevice\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtest_loader\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     42\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-8-6517550222a6>\u001b[0m in \u001b[0;36mtrain\u001b[1;34m(net, device, train_loader, optimizer, criterion, epoch)\u001b[0m\n\u001b[0;32m      3\u001b[0m         \u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mto\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdevice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m         \u001b[0mloss\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mbp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mTBPTT\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnet\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mnum_steps\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0moptimizer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mK\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mK\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      6\u001b[0m         \u001b[1;31m# loss = bp.BPTT(net, data, target, num_steps, batch_size, optimizer, criterion)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Dropbox\\repos\\snntorch\\snntorch\\backprop.py\u001b[0m in \u001b[0;36mTBPTT\u001b[1;34m(net, data, target, num_steps, batch_size, optimizer, criterion, K)\u001b[0m\n\u001b[0;32m     30\u001b[0m     \u001b[0mloss_avg\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     31\u001b[0m     \u001b[1;32mfor\u001b[0m \u001b[0mstep\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnum_steps\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 32\u001b[1;33m         \u001b[0mspk_out\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmem_out\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnet\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mview\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mbatch_size\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     33\u001b[0m         \u001b[0mloss\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcriterion\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmem_out\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarget\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     34\u001b[0m         \u001b[0mloss_trunc\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[0mloss\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\envs\\py367\\lib\\site-packages\\torch\\nn\\modules\\module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[1;34m(self, *input, **kwargs)\u001b[0m\n\u001b[0;32m    720\u001b[0m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    721\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 722\u001b[1;33m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    723\u001b[0m         for hook in itertools.chain(\n\u001b[0;32m    724\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-7-edf7cc0c04fb>\u001b[0m in \u001b[0;36mforward\u001b[1;34m(self, x)\u001b[0m\n\u001b[0;32m     13\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mforward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     14\u001b[0m         \u001b[0mcur1\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfc1\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 15\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mspk1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem1\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcur1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     16\u001b[0m         \u001b[0mcur2\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfc2\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif1\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mspk\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     17\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mspk\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcur2\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlif2\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\anaconda3\\envs\\py367\\lib\\site-packages\\torch\\nn\\modules\\module.py\u001b[0m in \u001b[0;36m_call_impl\u001b[1;34m(self, *input, **kwargs)\u001b[0m\n\u001b[0;32m    720\u001b[0m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_slow_forward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    721\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 722\u001b[1;33m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mforward\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    723\u001b[0m         for hook in itertools.chain(\n\u001b[0;32m    724\u001b[0m                 \u001b[0m_global_forward_hooks\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Dropbox\\repos\\snntorch\\snntorch\\__init__.py\u001b[0m in \u001b[0;36mforward\u001b[1;34m(self, input_, syn, mem)\u001b[0m\n\u001b[0;32m    141\u001b[0m         \u001b[1;31m# intended for truncated-BPTT where instance variables are hidden states\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    142\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhidden_init\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 143\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mspk\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfire\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    144\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0malpha\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0minput_\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    145\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mbeta\u001b[0m \u001b[1;33m*\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmem\u001b[0m \u001b[1;33m+\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msyn\u001b[0m \u001b[1;33m-\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreset\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Dropbox\\repos\\snntorch\\snntorch\\__init__.py\u001b[0m in \u001b[0;36mfire\u001b[1;34m(self, mem)\u001b[0m\n\u001b[0;32m     33\u001b[0m         \u001b[0mreset\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mzeros_like\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmem\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     34\u001b[0m         \u001b[0mspk_idx\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mmem_shift\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 35\u001b[1;33m         \u001b[0mreset\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mspk_idx\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mones_like\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmem\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mspk_idx\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     36\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mspk\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mreset\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     37\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 4. Results\n",
    "### 4.1 Plot Training/Test Loss"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "source": [
    "# Plot Loss\n",
    "fig = plt.figure(facecolor=\"w\", figsize=(10, 5))\n",
    "plt.plot(loss_hist)\n",
    "plt.plot(test_loss_hist)\n",
    "plt.legend([\"Test Loss\", \"Train Loss\"])\n",
    "plt.xlabel(\"Epoch\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAE9CAYAAACLPV+MAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAfeElEQVR4nO3de3BU9f3/8ddKCBcBbbnIwqIx3RhjwibAchEVijGAwQaB1kkFE0pphIIRqVzsZX5+HSkRb0DpwGSmXEQFWi9JBIKCylQtmC6IFqN1C4lklwAhmARQJAnn94clI02AzXXzCc/HTGfY3c/Z8z454Dx7djdrsyzLEgAAAIxxVbAHAAAAQP0QcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGCYkGAP0JJ69OihsLCwYI8BAABwWYWFhTp+/Hidj11RARcWFiaPxxPsMQAAAC7L7XZf9DFeQgUAADAMAQcAAGAYAg4AAMAwV9R74AAAQONVVlbK5/PpzJkzwR6lTejYsaMcDofat28f8DYEHAAAqBefz6euXbsqLCxMNpst2OMYzbIslZaWyufz6cYbbwx4O15CBQAA9XLmzBl1796deGsCNptN3bt3r/fVTAIOAADUG/HWdBrys+QlVAAAYJTS0lLFx8dLko4cOaJ27dqpZ8+ekqS8vDyFhoZecvudO3cqNDRUw4cPr/XY2rVr5fF4tGLFiqYfvAkRcAAAwCjdu3fXvn37JEmPP/64unTpokcffTTg7Xfu3KkuXbrUGXCm4CVUAABgvD179mjkyJEaNGiQxowZo+LiYknS8uXLdcstt8jlcik5OVmFhYVatWqVnn/+ecXFxem9994L6Pmfe+45xcTEKCYmRkuXLpUknT59WuPGjVNsbKxiYmK0adMmSdLChQtr9lmfsKwPrsABAACjWZalhx56SNnZ2erZs6c2bdqk3/3ud1q9erUyMjJUUFCgDh06qKysTNdee61mzJhRr6t2e/bs0Zo1a/Thhx/KsiwNHTpUI0eO1MGDB9WnTx9t2bJFklReXq4TJ07o9ddf1+effy6bzaaysrJmOWYCDgAANNj/vfGp8g9XNOlz3tKnm/7fT6IDXv/tt99q//79SkhIkCRVV1fLbrdLklwulyZPnqx7771X9957b4Pmef/99zVhwgRdffXVkqSJEyfqvffe09ixY/Xoo49qwYIFuueee3THHXeoqqpKHTt21PTp0zVu3Djdc889Ddrn5fASKgAAMJplWYqOjta+ffu0b98+/etf/9Jbb70lSdqyZYtmzZqlPXv2aNCgQaqqqmrQ89flpptu0p49e9S/f3899thjeuKJJxQSEqK8vDxNmjRJWVlZGjt2bKOO7WK4AgcAABqsPlfKmkuHDh1UUlKiXbt26dZbb1VlZaW++OILRUVFqaioSKNGjdLtt9+ul19+WadOnVLXrl1VURH4VcMRI0Zo6tSpWrhwoSzL0uuvv67169fr8OHD+uEPf6gpU6aoS5cuWrt2rU6dOqWvv/5aiYmJGjZsmJxOZ7McMwEHAACMdtVVV+mVV15Renq6ysvLVVVVpTlz5uimm27SlClTVF5eLsuy9Mgjj+jaa6/VT37yE/30pz9Vdna2/vSnP+mOO+644PnWrl2rrKysmtu7d+/W1KlTNWTIEEnS9OnTNWDAAL355puaN2+errrqKrVv314rV67UyZMnNX78eJ05c0aWZen5559vlmO2WRe7LtgGud1ueTyeYI8BAIDRPvvsM0VFRQV7jDalrp/ppbqF98ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAADAKKWlpYqLi1NcXJx69+6tvn371tw+e/bsJbf1eDxKT0+v1/7CwsJ0/Pjxxozc5PhFvgAAwCjdu3fXvn37JEmPP/54rS+mr6qqUkhI3YnjdrvldrtbYsxmxRU4AABgvKlTp2ru3LkaNWqUFixYoLy8PA0fPlwDBgzQ8OHD9e9//1uStHPnzpovmH/88cc1bdo0/fjHP1Z4eLiWL18e8P6+/PJLxcfHy+VyKT4+XocOHZIk/e1vf1NMTIxiY2M1YsQISdKnn36qIUOGKC4uTi6XS16vt9HHyxU4AADQJnzxxRfasWOH2rVrp4qKCv39739XSEiIduzYod/+9rd69dVXa23z+eef691339XJkycVGRmpmTNnqn379pfd1+zZs5WSkqLU1FStXr1a6enpysrK0hNPPKE333xTffv2VVlZmSRp1apVevjhhzV58mSdPXtW1dXVjT5WAg4AADRc7kLpyL+a9jl795fuzqj3Zj/72c/Url07SVJ5eblSU1Pl9Xpls9lUWVlZ5zbjxo1Thw4d1KFDB/Xq1UtHjx6Vw+G47L527dql1157TZL0wAMPaP78+ZKk2267TVOnTtV9992niRMnSpJuvfVWLVq0SD6fTxMnTlRERES9j+1/8RIqAABoE66++uqaP//hD3/QqFGjtH//fr3xxhs6c+ZMndt06NCh5s/t2rVTVVVVg/Zts9kkfXe17cknn1RRUZHi4uJUWlqq+++/Xzk5OerUqZPGjBmjd955p0H7+D6uwAEAgIZrwJWyllBeXq6+fftKktauXdvkzz98+HBt3LhRDzzwgF566SXdfvvtkqQDBw5o6NChGjp0qN544w0VFRWpvLxc4eHhSk9P18GDB/XJJ5/ozjvvbNT+uQIHAADanPnz5+uxxx7Tbbfd1iTvOXO5XHI4HHI4HJo7d66WL1+uNWvWyOVyaf369Vq2bJkkad68eerfv79iYmI0YsQIxcbGatOmTYqJiVFcXJw+//xzpaSkNHoem2VZVqOfxRBut1sejyfYYwAAYLTPPvtMUVFRwR6jTanrZ3qpbgnqFbht27YpMjJSTqdTGRm1L8FalqX09HQ5nU65XC7t3bv3gserq6s1YMCAmo8DAwAAXAmCFnDV1dWaNWuWcnNzlZ+frw0bNig/P/+CNbm5ufJ6vfJ6vcrMzNTMmTMveHzZsmX8PwAAAHDFCVrA5eXlyel0Kjw8XKGhoUpOTlZ2dvYFa7Kzs5WSkiKbzaZhw4aprKxMxcXFkiSfz6ctW7Zo+vTpwRgfAAAgaIIWcH6/X/369au57XA45Pf7A14zZ84cLVmyRFddxecwAABoaVfQW+ibXUN+lkGrn7qGPf87VC63ZvPmzerVq5cGDRp02f1kZmbWfO9ZSUlJwwcGAACSpI4dO6q0tJSIawKWZam0tFQdO3as13ZB+z1wDodDRUVFNbd9Pp/69OkT0JpXXnlFOTk52rp1q86cOaOKigpNmTJFL774Yq39pKWlKS0tTZLaxJfXAgAQbA6HQz6fjwsjTaRjx44BffvD9wUt4AYPHiyv16uCggL17dtXGzdu1Msvv3zBmqSkJK1YsULJycn68MMPdc0118hut2vx4sVavHixpO++lPaZZ56pM94AAEDTa9++vW688cZgj3FFC1rAhYSEaMWKFRozZoyqq6s1bdo0RUdHa9WqVZKkGTNmKDExUVu3bpXT6VTnzp21Zs2aYI0LAADQavCLfAEAAFqhVvuLfAEAAFB/BBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDBBDbht27YpMjJSTqdTGRkZtR63LEvp6elyOp1yuVzau3evJKmoqEijRo1SVFSUoqOjtWzZspYeHQAAIGiCFnDV1dWaNWuWcnNzlZ+frw0bNig/P/+CNbm5ufJ6vfJ6vcrMzNTMmTMlSSEhIXr22Wf12Wefaffu3frzn/9ca1sAAIC2KmgBl5eXJ6fTqfDwcIWGhio5OVnZ2dkXrMnOzlZKSopsNpuGDRumsrIyFRcXy263a+DAgZKkrl27KioqSn6/PxiHAQAA0OKCFnB+v1/9+vWrue1wOGpFWCBrCgsL9dFHH2no0KF17iczM1Nut1tut1slJSVNeAQAAADBEbSAsyyr1n02m61ea06dOqVJkyZp6dKl6tatW537SUtLk8fjkcfjUc+ePRs5NQAAQPAFLeAcDoeKiopqbvt8PvXp0yfgNZWVlZo0aZImT56siRMntszQAAAArUDQAm7w4MHyer0qKCjQ2bNntXHjRiUlJV2wJikpSS+88IIsy9Lu3bt1zTXXyG63y7Is/fKXv1RUVJTmzp0bpCMAAAAIjpCg7TgkRCtWrNCYMWNUXV2tadOmKTo6WqtWrZIkzZgxQ4mJidq6daucTqc6d+6sNWvWSJI++OADrV+/Xv3791dcXJwk6Y9//KMSExODdTgAAAAtxmbV9UazNsrtdsvj8QR7DAAAgMu6VLfwTQwAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGCaggDt9+rTOnTsnSfriiy+Uk5OjysrKZh0MAAAAdQso4EaMGKEzZ87I7/crPj5ea9as0dSpU5t5NAAAANQloICzLEudO3fWa6+9poceekivv/668vPzm3s2AAAA1CHggNu1a5deeukljRs3TpJUVVXVrIMBAACgbgEF3NKlS7V48WJNmDBB0dHROnjwoEaNGtXcswEAAKAOAQXcyJEjlZOTowULFujcuXPq0aOHli9f3uidb9u2TZGRkXI6ncrIyKj1uGVZSk9Pl9PplMvl0t69ewPeFgAAoK0KKODuv/9+VVRU6PTp07rlllsUGRmpp59+ulE7rq6u1qxZs5Sbm6v8/Hxt2LCh1vvqcnNz5fV65fV6lZmZqZkzZwa8LQAAQFsVUMDl5+erW7duysrKUmJiog4dOqT169c3asd5eXlyOp0KDw9XaGiokpOTlZ2dfcGa7OxspaSkyGazadiwYSorK1NxcXFA2wIAALRVAQVcZWWlKisrlZWVpfHjx6t9+/ay2WyN2rHf71e/fv1qbjscDvn9/oDWBLItAABAWxVQwD344IMKCwvT6dOnNWLECH355Zfq1q1bo3ZsWVat+/43Ci+2JpBtz8vMzJTb7Zbb7VZJSUkDpwUAAGg9Agq49PR0+f1+bd26VTabTTfccIPefffdRu3Y4XCoqKio5rbP51OfPn0CWhPItuelpaXJ4/HI4/GoZ8+ejZoZAACgNQgo4MrLyzV37tyaK1m/+c1vdPr06UbtePDgwfJ6vSooKNDZs2e1ceNGJSUlXbAmKSlJL7zwgizL0u7du3XNNdfIbrcHtC0AAEBbFRLIomnTpikmJkZ//etfJUnr16/XL37xC7322msN33FIiFasWKExY8aourpa06ZNU3R0tFatWiVJmjFjhhITE7V161Y5nU517txZa9asueS2AAAAVwKbVdcbyv5HXFyc9u3bd9n7Wju32y2PxxPsMQAAAC7rUt0S0EuonTp10vvvv19z+4MPPlCnTp2aZjoAAADUS0Avoa5atUopKSkqLy+XJP3gBz/QunXrmnUwAAAA1C2ggIuNjdXHH3+siooKSVK3bt20dOlSuVyuZh0OAAAAtQX0Eup53bp1q/n9b88991yzDAQAAIBLq1fAfV8An30AAABAM2hwwDX2q7QAAADQMJd8D1zXrl3rDDXLsvTNN98021AAAAC4uEsG3MmTJ1tqDgAAAASowS+hAgAAIDgIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhghJwJ06cUEJCgiIiIpSQkKCvvvqqznXbtm1TZGSknE6nMjIyau6fN2+ebr75ZrlcLk2YMEFlZWUtNDkAAEDwBSXgMjIyFB8fL6/Xq/j4+Avi7Lzq6mrNmjVLubm5ys/P14YNG5Sfny9JSkhI0P79+/XJJ5/opptu0uLFi1v6EAAAAIImKAGXnZ2t1NRUSVJqaqqysrJqrcnLy5PT6VR4eLhCQ0OVnJys7OxsSdLo0aMVEhIiSRo2bJh8Pl+LzQ4AABBsQQm4o0ePym63S5LsdruOHTtWa43f71e/fv1qbjscDvn9/lrrVq9erbvvvvui+8rMzJTb7Zbb7VZJSUkTTA8AABBcIc31xHfddZeOHDlS6/5FixYFtL1lWbXus9lstZ4rJCREkydPvujzpKWlKS0tTZLkdrsD2jcAAEBr1mwBt2PHjos+dt1116m4uFh2u13FxcXq1atXrTUOh0NFRUU1t30+n/r06VNze926ddq8ebPefvvtWmEHAADQlgXlJdSkpCStW7dO0nchNn78+FprBg8eLK/Xq4KCAp09e1YbN25UUlKSpO8+nfrUU08pJydHnTt3btHZAQAAgi0oAbdw4UJt375dERER2r59uxYuXChJOnz4sBITEyVJISEhWrFihcaMGaOoqCjdd999io6OliTNnj1bJ0+eVEJCguLi4jRjxoxgHAYAAEBQ2Ky63mzWRrndbnk8nmCPAQAAcFmX6ha+iQEAAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYJigBNyJEyeUkJCgiIgIJSQk6Kuvvqpz3bZt2xQZGSmn06mMjIxajz/zzDOy2Ww6fvx4c48MAADQagQl4DIyMhQfHy+v16v4+Pg646y6ulqzZs1Sbm6u8vPztWHDBuXn59c8XlRUpO3bt+v6669vydEBAACCLigBl52drdTUVElSamqqsrKyaq3Jy8uT0+lUeHi4QkNDlZycrOzs7JrHH3nkES1ZskQ2m62lxgYAAGgVghJwR48eld1ulyTZ7XYdO3as1hq/369+/frV3HY4HPL7/ZKknJwc9e3bV7GxsS0zMAAAQCsS0lxPfNddd+nIkSO17l+0aFFA21uWVes+m82mr7/+WosWLdJbb70V0PNkZmYqMzNTklRSUhLQNgAAAK1ZswXcjh07LvrYddddp+LiYtntdhUXF6tXr1611jgcDhUVFdXc9vl86tOnjw4cOKCCgoKaq28+n08DBw5UXl6eevfuXet50tLSlJaWJklyu92NPSwAAICgC8pLqElJSVq3bp0kad26dRo/fnytNYMHD5bX61VBQYHOnj2rjRs3KikpSf3799exY8dUWFiowsJCORwO7d27t854AwAAaIuCEnALFy7U9u3bFRERoe3bt2vhwoWSpMOHDysxMVGSFBISohUrVmjMmDGKiorSfffdp+jo6GCMCwAA0KrYrLrebNZGud1ueTyeYI8BAABwWZfqFr6JAQAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYAg4AAMAwBBwAAIBhCDgAAADDEHAAAACGIeAAAAAMQ8ABAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4AAAAwxBwAAAAhiHgAAAADEPAAQAAGIaAAwAAMAwBBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAxDwAEAABiGgAMAADAMAQcAAGAYm2VZVrCHaCk9evRQWFhYsMcwRklJiXr27BnsMfA9nJPWifPS+nBOWifOS/0UFhbq+PHjdT52RQUc6sftdsvj8QR7DHwP56R14ry0PpyT1onz0nR4CRUAAMAwBBwAAIBhCDhcVFpaWrBHwP/gnLROnJfWh3PSOnFemg7vgQMAADAMV+AAAAAMQ8Bd4U6cOKGEhARFREQoISFBX331VZ3rtm3bpsjISDmdTmVkZNR6/JlnnpHNZrvox50RuMaek3nz5unmm2+Wy+XShAkTVFZW1kKTtz2X+3tvWZbS09PldDrlcrm0d+/egLdFwzX0vBQVFWnUqFGKiopSdHS0li1b1tKjt1mN+bciSdXV1RowYIDuueeelhrZfBauaPPmzbMWL15sWZZlLV682Jo/f36tNVVVVVZ4eLh14MAB69tvv7VcLpf16aef1jx+6NAha/To0db1119vlZSUtNjsbVVjz8mbb75pVVZWWpZlWfPnz69ze1ze5f7eW5ZlbdmyxRo7dqx17tw5a9euXdaQIUMC3hYN05jzcvjwYWvPnj2WZVlWRUWFFRERwXlpAo05J+c9++yz1s9//nNr3LhxLTm60bgCd4XLzs5WamqqJCk1NVVZWVm11uTl5cnpdCo8PFyhoaFKTk5WdnZ2zeOPPPKIlixZIpvN1lJjt2mNPSejR49WSEiIJGnYsGHy+XwtNntbcrm/99J35yolJUU2m03Dhg1TWVmZiouLA9oWDdOY82K32zVw4EBJUteuXRUVFSW/3x+Mw2hTGnNOJMnn82nLli2aPn16MMY3FgF3hTt69KjsdrskyW6369ixY7XW+P1+9evXr+a2w+Go+Y9eTk6O+vbtq9jY2JYZ+ArQ2HPyfatXr9bdd9/dfMO2YYH8jC+2JtDzg/przHn5vsLCQn300UcaOnRo8w58BWjsOZkzZ46WLFmiq64iSeojJNgDoPndddddOnLkSK37Fy1aFND2Vh0fVLbZbPr666+1aNEivfXWW42e8UrTXOfkf58rJCREkydPbtiQV7hAfsYXWxPItmiYxpyX806dOqVJkyZp6dKl6tatW9MPeYVpzDnZvHmzevXqpUGDBmnnzp3NNWKbRMBdAXbs2HHRx6677rqalxaKi4vVq1evWmscDoeKiopqbvt8PvXp00cHDhxQQUFBzdU3n8+ngQMHKi8vT7179276A2lDmuucnLdu3Tpt3rxZb7/9NuHQQJf7GV9qzdmzZy+7LRqmMedFkiorKzVp0iRNnjxZEydObJmh27jGnJNXXnlFOTk52rp1q86cOaOKigpNmTJFL774YovNb6ygvfsOrcKjjz56wRvm582bV2tNZWWldeONN1oHDx6seYPq/v37a6274YYb+BBDE2jsOcnNzbWioqKsY8eOtejcbU0gf+83b958wRuzBw8eHPC2aJjGnJdz585ZDzzwgPXwww8HYfK2qzHn5PveffddPsRQDwTcFe748ePWnXfeaTmdTuvOO++0SktLLcuyLL/fb919990167Zs2WJFRERY4eHh1pNPPlnncxFwTaOx5+RHP/qR5XA4rNjYWCs2NtZ68MEHW/wY2oq6fsYrV660Vq5caVnWd0Hw61//2goPD7diYmKsf/7zn5fcFk2joeflvffesyRZ/fv3r/n3sWXLlqAdR1vSmH8r5xFw9cM3MQAAABiGj3wAAAAYhoADAAAwDAEHAABgGAIOAADAMAQcAACAYQg4APivdu3aKS4uruZ/GRkZTfbchYWFiomJabLnA3Bl45sYAOC/OnXqpH379gV7DAC4LK7AAcBlhIWFacGCBRoyZIiGDBmi//znP5KkL7/8UvHx8XK5XIqPj9ehQ4ckSUePHtWECRMUGxur2NhY/eMf/5AkVVdX61e/+pWio6M1evRoffPNN0E7JgBmI+AA4L+++eabC15C3bRpU81j3bp1U15enmbPnq05c+ZIkmbPnq2UlBR98sknmjx5stLT0yVJ6enpGjlypD7++GPt3btX0dHRkiSv16tZs2bp008/1bXXXqtXX321xY8RQNvANzEAwH916dJFp06dqnV/WFiY3nnnHYWHh6uyslK9e/dWaWmpevTooeLiYrVv316VlZWy2+06fvy4evbsKZ/Ppw4dOtQ8R2FhoRISEuT1eiVJTz31lCorK/X73/++xY4PQNvBFTgACIDNZqvzzxdbU5fvB127du1UVVXVNMMBuOIQcAAQgPMvp27atEm33nqrJGn48OHauHGjJOmll17S7bffLkmKj4/XypUrJX33vreKioogTAygLeNTqADwX+ffA3fe2LFja36VyLfffquhQ4fq3Llz2rBhgyRp+fLlmjZtmp5++mn17NlTa9askSQtW7ZMaWlp+stf/qJ27dpp5cqVstvtLX48ANou3gMHAJcRFhYmj8ejHj16BHsUAJDES6gAAADG4QocAACAYbgCBwAAYBgCDgAAwDAEHAAAgGEIOAAAAMMQcAAAAIYh4AAAAAzz/wFNdUgoDnnG6wAAAABJRU5ErkJggg=="
     },
     "metadata": {}
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "That's it for static MNIST! Now let's use ``spikeplot`` to watch in real time how the output layer responds to a few different samples."
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%% md\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "from snntorch import utils\n",
    "# Let's just test on one single batch\n",
    "mnist_anim = datasets.MNIST(data_path, train=False, download=True, transform=transform)\n",
    "mnist_anim = utils.data_subset(mnist_anim, subset=78)\n",
    "anim_loader = DataLoader(mnist_anim, batch_size=batch_size, shuffle=True, drop_last=False)\n",
    "\n",
    "# Pass anim_loader into test\n",
    "_, _, spk_rec = test(net, device, anim_loader, criterion)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "\n",
      "Test set: Average loss: 0.15009622275829315, Accuracy: [123/128] (0.9609375)\n",
      "=====================\n",
      "\n"
     ]
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "source": [
    "print(f\"Data Size: {anim_loader.dataset[0][0].size()}\")\n",
    "print(f\"Target: {anim_loader.dataset[22][1]}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Data Size: torch.Size([1, 28, 28])\n",
      "Target: 6\n"
     ]
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "# spk_rec is a T x B x N_outputs tensor\n",
    "# but we only want a single sample, so T x N tensor is input to snnboard\n",
    "torch.stack(spk_rec, dim=0)[:, 0, :].size()"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "torch.Size([25, 10])"
      ]
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "source": [
    "spk_results = torch.stack(spk_rec, dim=0)[:, 22, :].to('cpu')\n",
    "print(f\"Total number of spikes at t=T for a single sample:\\n {spk_results.sum(dim=0)}\")"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Total number of spikes at t=T for a single sample:\n",
      " tensor([0., 0., 0., 0., 0., 0., 8., 0., 0., 0.])\n"
     ]
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "source": [
    "print(f\"Target: {anim_loader.dataset[22][1]}\")\n",
    "plt.figure(facecolor=\"w\")\n",
    "plt.subplot(1,2,1)\n",
    "plt.imshow(anim_loader.dataset[22][0].reshape((28,-1)).cpu(), cmap='plasma')\n",
    "plt.axis('off')\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Target: 6\n"
     ]
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAKYAAACmCAYAAABQiPR3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAE7UlEQVR4nO3dz4uVVRyA8a9TYhBTtBCnWiRDboLAmXBT2excWDEQthIqCAvcVBtpSMLFwKQbp3aWAzEhxBTFQM2iTWlkhKgF0UaQcRGVLRQGyRC8/QHne6Z7596587zX57M892V8B5974Mx5f2xqtVqtkGCGNvoEpIxhCskwhWSYQjJMIRmmkO5e7cPhe2b6dR66Q63cnErHnTGFZJhCMkwhGaaQDFNIhikkwxSSYQrJMIVkmEIyTCEZppAMU0iGKaRVL3sbBIs7b6fj46fnirHpymV+J7Zc7uk56f85YwrJMIVkmEIyTCEZppAGflU+MTufjq/0+TzUGWdMIRmmkAxTSIYppIFa/Dx168FibGXsVnrshYlXizG3HjmcMYVkmEIyTCEZppAMU0gDtSo/tOv3to9dvlKu4CP+6t3JqCvOmEIyTCEZppAMU0gDtfjZ/dpSMXajcuzC1eQ7ubm356O1c8YUkmEKyTCFZJhCMkwhNXJVnl0QHBFxY/8fxdjwxXyp/cPm8lhxOGMKyTCFZJhCMkwhNXLxM7vvfNvHXjq+Zx3PZO1e/3c0HX9spLaJWnrymYvF2MmFp9Njm3YHqDOmkAxTSIYpJMMUkmEKqZGr8q2j7W8nnj0zVvmkf3dEZivwoz++mx5be9ZSuw7Pf51/kLwqhrxSd8YUkmEKyTCFZJhCauTip2mm3v6sGOtkkZM9ZDYiYumnR4ux2qJqZu6DYuzEwefaPod+c8YUkmEKyTCFZJhCMkwhuSrvodrFv3cd+ajtn/HF9sPF2BvXKtunyZbi1OLjlXO4UA4ebPu0+s4ZU0iGKSTDFJJhCsnFTw9lW4812SInYpWFzjp4/4FtG34ONc6YQjJMIRmmkAxTSI1c/Px9OX8+5kgytv2Ryo1r17r7TmbP6Ex3VyLi3lPlsYQFBpkzppAMU0iGKSTDFJJhCqmRq/I3P38iHf80eTzK+Om5/Ifcf6Crczh16Ku2j/3+w71d/VuduG9H/leIoeTtHeS/DDhjCskwhWSYQjJMITVy8VN73V629Ze9xi8iYnHn7WJs8uf1+Z4uX8m2ULtfeGTXU9Z+39pjZqicMYVkmEIyTCEZppAMU0iNXJXXfPJOufJ8Yf90emy2Vbl8ZDw9dua9F4uxTh77snA1+f6XO4RVtbsZX/7yaDmYbD1GRBw793ByDu2//aPfnDGFZJhCMkwhGaaQNrVarVbtw+HkNW9Nk209RkRMzM4XY92+Lq+m9jiYzOQrS8VY7e7LzHTl/4z6er6Vm1PpuDOmkAxTSIYpJMMUkmEKaeBX5TXZs4dm951Pjx2pvZy+T7ILoCMinj9Q3ulZu4iaylW5GsUwhWSYQjJMId2xi59uXf/u42Ksky3NP196Nh0/e2asGCM/yqVbLn7UKIYpJMMUkmEKyTCFNFB3SfbTpeN7irFOti63juZbh5PJ+G/JXZoR3It/e8EZU0iGKSTDFJJhCsnFzxrtWnyoGDtX2Wbc8dY3xdj1ys/95dtyS/LXoX86ObWB4IwpJMMUkmEKyTCFZJhC8kJhbSgvFFajGKaQDFNIhikkwxSSYQrJMIVkmEIyTCEZppAMU0iGKSTDFJJhCskwhWSYQjJMIRmmkAxTSIYpJMMUkmEKyTCFZJhCMkwhGaaQVn1EjLRRnDGFZJhCMkwhGaaQDFNIhimk/wDBQN7aPIW+KQAAAABJRU5ErkJggg=="
     },
     "metadata": {}
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "source": [
    "import snntorch.spikeplot as sp\n",
    "from IPython.display import HTML\n",
    "\n",
    "fig, ax = plt.subplots(facecolor='w', figsize=(12, 7))\n",
    "labels=['0', '1', '2', '3', '4', '5', '6', '7', '8','9']\n",
    "\n",
    "# animation\n",
    "anim = sp.spike_count(spk_results, fig, ax, labels, animate=True, interpolate=5, num_steps = num_steps, time_step=1e-3)\n",
    "HTML(anim.to_html5_video())\n",
    "# anim.save(\"spike_bar.gif\")\n",
    "\n",
    "# final count\n",
    "# sp.spike_count(spk_results, fig, ax, labels, interpolate=5, num_steps = num_steps, time_step=1e-3)\n",
    "# plt.show()"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ],
      "text/html": [
       "<video width=\"864\" height=\"504\" controls autoplay loop>\n",
       "  <source type=\"video/mp4\" src=\"data:video/mp4;base64,AAAAIGZ0eXBNNFYgAAACAE00ViBpc29taXNvMmF2YzEAAAAIZnJlZQAARWltZGF0AAACoAYF//+c\n",
       "3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE2MSAtIEguMjY0L01QRUctNCBBVkMgY29kZWMg\n",
       "LSBDb3B5bGVmdCAyMDAzLTIwMjAgLSBodHRwOi8vd3d3LnZpZGVvbGFuLm9yZy94MjY0Lmh0bWwg\n",
       "LSBvcHRpb25zOiBjYWJhYz0xIHJlZj0zIGRlYmxvY2s9MTowOjAgYW5hbHlzZT0weDM6MHgxMTMg\n",
       "bWU9aGV4IHN1Ym1lPTcgcHN5PTEgcHN5X3JkPTEuMDA6MC4wMCBtaXhlZF9yZWY9MSBtZV9yYW5n\n",
       "ZT0xNiBjaHJvbWFfbWU9MSB0cmVsbGlzPTEgOHg4ZGN0PTEgY3FtPTAgZGVhZHpvbmU9MjEsMTEg\n",
       "ZmFzdF9wc2tpcD0xIGNocm9tYV9xcF9vZmZzZXQ9LTIgdGhyZWFkcz02IGxvb2thaGVhZF90aHJl\n",
       "YWRzPTEgc2xpY2VkX3RocmVhZHM9MCBucj0wIGRlY2ltYXRlPTEgaW50ZXJsYWNlZD0wIGJsdXJh\n",
       "eV9jb21wYXQ9MCBjb25zdHJhaW5lZF9pbnRyYT0wIGJmcmFtZXM9MyBiX3B5cmFtaWQ9MiBiX2Fk\n",
       "YXB0PTEgYl9iaWFzPTAgZGlyZWN0PTEgd2VpZ2h0Yj0xIG9wZW5fZ29wPTAgd2VpZ2h0cD0yIGtl\n",
       "eWludD0yNTAga2V5aW50X21pbj0yNSBzY2VuZWN1dD00MCBpbnRyYV9yZWZyZXNoPTAgcmNfbG9v\n",
       "a2FoZWFkPTQwIHJjPWNyZiBtYnRyZWU9MSBjcmY9MjMuMCBxY29tcD0wLjYwIHFwbWluPTAgcXBt\n",
       "YXg9NjkgcXBzdGVwPTQgaXBfcmF0aW89MS40MCBhcT0xOjEuMDAAgAAADh5liIQAL//+9q78yytH\n",
       "C5UuHVl7s1Hy6Ely/YgwfWgAAAMAAAMAAAMAAQ+YOPKPYkmR6AAAAwACfhlwAPu6vgc7gA6wEvCa\n",
       "G8sbj18IJ+q8NDNsdunk7y9diOg+/i/31pQZXAX2SIegfQJpJoThIClKEcoy81R+PBLUATsMqWIJ\n",
       "TRCpkGgfdCbkzUg1pB3iAGi40jde0L6J1eENSawmAsv7cxMoJy+TAZUxXDFb/dc+v5E+dEpVLwz3\n",
       "R0t0BgRIE+EGwPHbYhrd50PoZmgtbxWxa3XR7YMRYsVV1YAMjSSmYNzDVHdXrpEoeAzp/6iynKZE\n",
       "eO0DHBEyi1QzVcrkN2J6geXZnMH7v7ziKZkB2zKoFpCWVkLIvfj0BZ0IJTE9RwzOsX7fzRfnwAXO\n",
       "4A+lCiCG0BdY3W5V+ONS3ZuNZNxJPYHyAXXe8u7t+tv/nP/YD+kH3cTx9Ajg0GthtFtUQsL9mXgR\n",
       "n9j555Km3TfeY3XaYnWCl+R/SB95Lj8L1f1+nbWdFpO00907Lzr9irwxun4gdH2ko1C8IY1CDLSd\n",
       "htSYXR6+349/SWbrHBR8JzE13fKfQJcYgS3CevMbz+WLcMpY42qMQtMmWJMyv8B9aaQ9yIYBw9Rj\n",
       "LwS1ZlVqvbH+lUR+bqYBKAa7JoVMZJJHGcW+IMEQDG6+euFzmmXX8IB1kSuQnQusXox7oHa+KoAK\n",
       "WJ8FWojQMnZfLxvriex4Hr4h5XwSRDbG/WX8dUAqDnoUw/AO3R+26jw1uoaqlwOw7ORtgT0qUwoY\n",
       "RecZnzFRHLZSMIZSC1Hf1woxJYf5XjAroSmQC9SlzQUbbyAdmwi8AIBUQ1Omqw5Wm0dMrfDNxMhk\n",
       "CwZM7NJxwEmnjwm1iljNDe0Mit8zP5HZJ67TtSNQ+EWMEbLMZ2XlzyoTQAHCWi7CGC/qZfp0Hgxx\n",
       "d3Y22zdalWwhcPrTYg3uSWgoGPJmYeNPE/l3kJbXeYlLhWluh9P+PzNqlYo1bN9Qjmecb/yjN46i\n",
       "GbNiV+FC+C0nmZscClKNvm1IyAUGMYMDZldEy6926ho4I0Ndro72i+aZV082ZwtgguCXer9tVJwC\n",
       "D3z6bwyq7Qgd1uvCC40gJ45UlmCI1hdO1fE4tNmcw5mnqjauumv995b0axEEhvUO3IsOMeKTKuCn\n",
       "Y/dKjNdDnuVOeECTmCzJo7TrwFn2golQajsblyr6gkuiC/FB5ks+mu3gHUfJsHRHRr0f1indU4Na\n",
       "u2WI7RmdxINCnOxm2L+8PW+cjW37Dc4QXsYnA5sXu4diERiimb+0XN6QBBFMHtSTJbx0VUzINHt8\n",
       "FeHJxbMcRab8DToOkvP3IY97Hkjwio4sZuJ6dC0FlVYV+bsa8rzsfj7KixwTN2hygdPYxp4IZle+\n",
       "DrbsNm2XgqFJsanncRJScDu1iT0z46S1a3bt7VI7zHAn7y4aZkDd4GpBo/WuAt8EOZOcnkt3L3Je\n",
       "vUxgb1DfKA+05Yg8z2qwpd5izQZ+OikXhcPwpmXuztVAfq0cWSVwi15YNh/RrTVmx+RYRvf8mUdq\n",
       "hFvKd+xafluFHYumnW5wQqBiR789MBSKCXQ5liZnNWB/5QZ/OaXY+oB63qpFB8+yyh/VKtJXgDrX\n",
       "8ESdv+P6/bcuL9MDbR4HYtoCuonQytQhuV0QTivqfwfmJT4mhdOjHO1VxWsdGd2rE+myUIOObTxx\n",
       "2K8EqR9QeULdYlFTpEcEo8gWLIpZDkPWV8Vv1hi+1A7olDLj/EmTY1EvBq9XYyyKMt+KlKcBO6ET\n",
       "qsT3eB8Vg965g18hiWr4ZexP0L3zmoiVYE9sVrJNDIqVI8GRhglUrdajyfUg0L7pheFwTX54os6s\n",
       "QvUCib47Ebtp9VkfRmKkSG0nRPcPhIaMexMnCUHR9kaQBQ/NBN4OQxVT8tujFzAEE612ZrMY57vo\n",
       "PDiaBcxFv9JZ8jTxROqa0DQKBqC0Ga0Dof3A5Se7/s//13AB68OPYgokeK5Fq6vaTF2wDMyAKH3l\n",
       "W80APIzprLIS4XW9YdodzV0goaHqTW7HSXreBkcmjI+uk3VGjwhTfpXHbpmge21aK56MsKCwr7hd\n",
       "MuCsS71Jwmmb2tiNtyk8qXZApM3IVJGMeuocili6iyxGxfu6wm9Rob5iIiUxm39CxRh0otNliI9z\n",
       "+NN9+AmXwK9Az/6n3B9NygVVb32gvDwqhSvOG7ChIWoclsvFCu5z16LZwrYifFnWmh1cG9ozS5zt\n",
       "OxE3o15N3wnqUf6aKkPWhaQuskiGT7RblW9D1gQ+S94BierpO6d/31mAna7hBefy11AYh2yik2wK\n",
       "F/4+jMVHwpB6IIuqJ8/7KNFik9nAvkE9NbvV4eCLwbmErnGuCdOZkcCWtYTRGc23ALS1T6vzBcmV\n",
       "PEBpWBL6vljQgc3mwgO0+oZ/ulgSn0Kau1kTsG1QYtYjmIwHql2pdlfWHRA+ThNnqU/GFLkmUYMN\n",
       "6aoks+e/5zSWw/0CC5PzHYjhf/i2M3b0qQk+o2rs0d5Q5MkCT7vasuPv4Y2H3DXBE27/QbksW5Y+\n",
       "SQw5mPoMIwYecehEK4r1IE2b9fzajy8TEHZ2QuZDiR7cR7fVlhF1XHBXo82mFbCIcUdiEIPQt9sN\n",
       "19GAhFVblrhpcUpsA5u8UuIydM4RD9aJNJpIQoBSOf2/lkOaa58HSrv5i6Ywe1Ahyqt1AL15MLCW\n",
       "YjDL7SK6+KMexuVFZA0Tl9sOCt4JATXe4nwd6TuMlk6cyynMnjZPXFZVBMnz9QL/QKGpRuzQLP/M\n",
       "/2qbQORSvH9qTZCexVPt0qU2Vp5aTLBNHtP58H3+oZ5rwGSre97Y7AGjUgAB/1w+vnsKHKLU2sWi\n",
       "frC/XAcyhFb743b5/eTsBONsxq8hBlJ1+Z8aJGKLc4Ul1Wf2b/EvtA07tZP7TQXN1EFca3X0il54\n",
       "p80UJjyTKVxG2PINp9D/WI6SnTlBpyfHUiTor5jauXbF9DCZ3j1vIy2b2nSWtk4OH3uR55/lEyU0\n",
       "mQejU5/8RyH1CtZqLJPikROCE/VVsWfK/lpDnizct/+p5bsKGbC+TPEdZUjhDe2D0MOtlhjZDikl\n",
       "uK22SQ/kf/QfeA1NbGWs7lFl6BkdslBVVbFCXbEePz3bWLTgC7mgnknFd49bkF3PqBM+QZUNwrEH\n",
       "XY0bu6gZK3YrcGJCH9+blZzLjD5d218DKsiZpvY1jAXuwpmbxH7SbfWc8YQxcx7z4GZ4LUHPu9SH\n",
       "23qMjnOBiogMYQhalMeEUOFROAiv2gcsGPWEXqI39HQRIv6VYWTa5UshylU/aw3etqB65xuLrnDc\n",
       "03fII5sVSh3pGVcObZgWOaQg2dinTJiCVER7EphWL6jQXev4/42jrH8hAiGa1bJjXChygDGwovz5\n",
       "SdcsP8KV3C2Gmonuk82Al6e6J6gzjAQEYBynTnQFlq1avSeKMOC3WA/9qZWh9Uoc6dfWWM60ZxVh\n",
       "EBDslCTCNSFJQSMzXh1qlkb293JJYZO1z5jKMxHrB5Z3w62rytbmSOtAI5w6BkBCv7C9RiCHNDaT\n",
       "MJJyUvxMpKyzGaXnF4xayelrFq2yw2+jMUKKGnO0AqWCkMdvaOZ3RvuUO8/cTQ9qOkFBfSrg/lWy\n",
       "jU0Qb7RzgUD8sQajqlpjNhWXMq6zb1vRHSqRN6DPXBbOzlTv11++U9QRe3h97c0EF+YwmlG3IPjD\n",
       "dv7UiOoOK0wEPHp0m7DCIg94DJEhncHi60/6K3i0JB5LqkiqcN3TK+PY24cRfwGqJsX/0Z/giQzf\n",
       "9JBMhj1yB7KQTo9HskujOTuKfK/FESHXu8Q9PMrCXxpHx87HKcc5W8nETAOt6eZSXLYe6CqIOqYI\n",
       "1IQ9AVPo8i77f+ozxTFpnTAvwWe6e+MyPv/IaSWZ1YlZKMP3mffOdMBq1Wxn2Arm/YcJxSczd8u0\n",
       "okvk0mTHkRYPFw6C5rR6+gWhyvYlGdYdIZkGhSagQLrHnrG+evibLaNRtvdvtXcvDLmxIzhI73nI\n",
       "Nu917T2E3Pr7ckMa443z3BPluYJdCSIgsq7yXIfJtAUj4cDxi0s/T5d2TnhCiDH+MLPxoRyEYS19\n",
       "sz8F7Ji4Soy7b3I6bX1aY8mq7cCqIt5MI4bR0wZPlMAMWIyL4pVqvPC1e8FCullbKjc/3+0U3RrD\n",
       "BsPwBrl7YPONOZKmaA5O+ltbdPMzY1GVP9cJR3qi+LTnnE5KqW7Fx2yLd90bAe4Okql/dq/9PhKg\n",
       "b7ZoVY0AKVXSbJ640h+v6Jn7rhA7HQjAKHDDVst1DHyIKkm//iwNCSOnVwsjfT70XaZRrl3+3QzY\n",
       "uSi6TpZj9wvIpv9aNrOti4PbGQ+hFAPyAkOSE3d17xfJyPgxrvUBCZIJr+nGOyiZSrDVasQIyf4T\n",
       "isPASAAcgnT4ECUHBZFMnOo/IVg9YOcaTSvyCAhkeVvHFrZ5NEQ5Wr/EcnURwkFUFrP0AaFqif1U\n",
       "x5Gp189Qxp+InseUG9Ktl2XEEOKUwlcDOXO0JqdAUl+MPv8GRDdZE3gO+LMKqQKx/Fnf+kbK0ZsZ\n",
       "AroKtnENSS/fjXZ53y+vRMaJxFF3L/D6J9zkX5//9FnlmcuIXeeY+95BgU+wobZi4OrwSnyFxzeA\n",
       "AoiTtBV+AFsKV7BCKSB0vZbcr1SMAfHMJa0OTDgdnm5r1Np+ZB01UYCS1GWQzVBhHC1+QXwHKhep\n",
       "I/mISLgFDx1SP279qvG8ajXI/tb0jNpoMGwB6X1vFiqCyOpCQ9nBrTXmVj5jwaieqMqoQnZhJ0d9\n",
       "VsKSs0xte9kAWJ74ZBjMzT9H11spcTp6LNdFY594SiX5cgpmfZ+CvVSRxVX9Vd+yW5gEGI/VoFYk\n",
       "cUAAA6IXGYCFgQAAAzBBmiJsQv/+jLAAZbWnwNP0Co4qr//f8CFhB+PvRY6YpB3yGpB556i3atPH\n",
       "wC0f+gONixdi5TRuo7uriM2bFY+dKPLquyUmLRfzQAN1y1PbdsgIyB7/wuapTBv534d8JkoZJV0f\n",
       "pm9OYE3jrEb6Y5mCpL8pdcm5Z4IdZwLqBkwX056oJLMgTZEtZETtG14djW7M6nNvUB65De72aiEk\n",
       "6E7jRCfpejtnlyNAH13443SdAd3eAAw7bQpe3+Hvu4pTOjRNiaQlRpsMI3LOrG+FEfQvRR+lBfDY\n",
       "suzYvhBsHXznay1pI2Tyhlcs9rmTege7T4EAvAbxlKjs8u40CebsVVN0P2rjsmT9/04uAJekmocF\n",
       "aKPK7agoUtBRYSyifB4VQ/YKdvwNiqqXLuZ80Qu+xiCWuRPJkmeVCg+JIE4+97ZQ7iIUgPiGL9oT\n",
       "syFCP4FmtEmLDfA8lXsLYcXScevawBEnGuV5HaRVwRnYhuvu438S2U600xLB591NtPr8KpgCbuxY\n",
       "i89TJyb79+jjx4kbz4K1URxwZHv129Bo7jsELtueLxBlnVWe/Pifteu7uTDdlOgToSBkSKeWz/Fb\n",
       "5LtNIJCis8clAsRa+NLgPsHV0e3iVrFp0oZprh9AInqEupyATLG+U89FgD9AoFMKfNopl9U8s1Nt\n",
       "KvGuQzCLJ+D2PluJabKa5o0kAE29ZG4SwZ0TKGNWoIM0WOjwB+9Pmn/RZWfRzN2+0emXC+A+t3hR\n",
       "4niBuqtr1q88ck76KEESE8Qy3pogtUq8l8YRkr9WpAX3ULLR1VR+dTzAaeT7NzzU6zKefyqvN6Ms\n",
       "EWz3GWNCEjKOq36NzstukWXnNARrmskMPQqIacya5xBzR5YJR+Rh4kxf2Mv9Qk8MOwJQiJvzDjG3\n",
       "VCDwQ8B8p6UmliJLSVzeSazEvT3fvwgavSwL/35hkXVUl504OkD5qvlQTsoAlvlIDTtetXPTS0SD\n",
       "YMEMcbG8U2zZULVWObUO28Ir3X9frRr/3kO3u1KX3A+zcFIiGEYd3EmDko7HCh6ub7pCQpHNymhu\n",
       "Q6V6gZFYyjuF01RuW6y9AMmDmvk7aQ/02tKFv9xcwIAAAACdAZ5BeQj/AB/BE+AImC5nMOhwH1iy\n",
       "wvEYmCk++Eoa2Lxgr2I5w2y9YYdoV3pOudtKAQanMHHetNPBuZM6PK7GhLC+ZqKExaJotCKBFhLQ\n",
       "3+A7ohyG1xkcnLZVNxl8Rtld+lrJ4vCO6VOG1/m1v9x16Ra3lh1j0zKAkbca8mpqeeKf5JcuAJmy\n",
       "KO2gcivxvQbDJMGupy7CtAFHL4An4QAAAgJBmkQ8IZMphC///oywAGUdmQEkvgP+OMe9oZZD65im\n",
       "HcyiK5qdbaussHALpkCkl9w6qeWtk6qNYADNsLVLYRZSymwvMaN/7QGvjsGmrDOAnQ+Y/zKqvtQa\n",
       "Fi444xLZqN8rq7yHkxhQvUaBpkt0gjwUKRm2uiVWOX5jAI5mOQhm71u9kXofeOndkpm5Ff6zeNPe\n",
       "s5zY1X/jwOW+oO+VtXS/14l9u6EikAY02dqxtvQOv7OFUEX1qmAWqvryrpvgFoSq2vaQNVrC4KRT\n",
       "sxlAP9Yq0YiBfflaA4c0uvYoDFyDACFXDrlMfBLoI2PtNk1hBwdelTBA5U/mP3kgh8iY/Y1SMaQs\n",
       "05i3/UchWIfYI31wFNZkqt+Mtuts97unxf8HlZSNprjE79iS+ZXeYAsY2uSJ02R+zh4t3gPMlOPn\n",
       "F3IFSF9i0Aqx1sBYB2rORoEumI8Jj3tl0aR+Axm2DBm/MZ93nQxC8ni+LzqrjTBp7xwzxoHkCgzo\n",
       "W2p3TnCuD4+A2SAerjx1iaWJY4xjOOCqrGsm5Xagc4tsYA/HC64DnPE7cEv48IAOBdSk7fhjNZvC\n",
       "d4xSiV7ZAavveEizPl3d/etvczTgqC1OftWRX/sumpsuoxN1iU1XLbv1208lX27YgbfpB53438h0\n",
       "Kc3lHD+aEnHxICvn7epAUkgePz0h85H52CsfAAAArwGeY2pCPwAfv+NdfAAH9LqnJlTPHGEuTi1P\n",
       "4ONbVE4arcTisdQiksCewxJGYTBX8mY91VfzU2b+vOs2EW67U5wDkPtst1AafHfWLiOYLcYFu+VC\n",
       "oXgyuu5oFvzP9+UIUyegXGjJqXjXqg8wVRaKx9DAI7QHv5UFjShBFwg111mx2gcMzGvFvfwel8Eb\n",
       "nay1M6zFaeghQamzxJSRcW4X3awh5TJn/TZ2jlbjeIcgBB0AAACjQZpmSeEPJlMFPC///oywAGAr\n",
       "VgABOqryjYdWRs57OQfMGF3Q2RABY+GOzr/mshhWLd1KdabAYGamcHq5c4O46JNWKMmJvnEhpmz0\n",
       "ZoncJtsCVjNHPOBxo6MRcokh2bBenmx/U6LU97w+BDQkziAfw+UL5YJZoeb3OEhxHoshbMbczvF6\n",
       "Nn5Xas/wv8CGT7QSHFmjtFmKCL8xpxATkubxs7AMCQAAAF0BnoVqQj8AHlyO5AAECfUn/BKBlTzE\n",
       "0yWlcsRLPuMsH2NeSpfQpDqMYancnJeyeorYjXVEZ03g6I5zGh4t6ME+vDlknXogBUboy2pKKxLk\n",
       "tjjnPMazb5JLT72YB+UAAACTQZqJSeEPJlMCF//+jLAAYL1uq34GhAAC1t2G7q/CGai8lmtOG0Hj\n",
       "l5+GNyJtYM1QvwilBoP7jYNkt+9scnvtY81s1eOGwki1QZltmCZH5y1t3VFwSeX5o140U88a6hZf\n",
       "e6SCJmNSAQW87J/HaQGAOWkRXMccABUcq+/L0u9NgfSg45HUebY63xVicYw7vWkKBbCxAAAAaUGe\n",
       "p0URPCf/ABnBODxb1SFY0AB+AQB52S1Dy0lEfCbrtTH0mZ+Ban9GHrymwQAaj/V7Yapoqbpp5iHl\n",
       "56dF+pZmWRgiQ3bPvjAOjDj0VFkVHyWKMwXvNHSIgBHJ3JevlRZzkJuK0KC7gAAAAFIBnshqQj8A\n",
       "HbcJDUU8mUAK3+J1N6ENXsCp8fZHzmaimt0QDy9VDJzWjXryQHgMs4DBciu4QjOGtNYvDmyGZtAd\n",
       "7MOmAcDOUWXEyOdF2N7EDg44AAAAU0GaykmoQWiZTAhf//6MsABeOZUmo1aDmovhWAr16nSa3I0O\n",
       "x0GJ3r9VtA7A9AZ7xQ2nHK1nhSHFeb9IhLM7B/fXRjPglhk1Dr7RlqC5rNqkVLKBAAAASUGa60nh\n",
       "ClJlMCGf/p4QAFqrez4gBJLm3zmBhNRq5uARSeqpsSQeAIIxEDUDpkpO5zyFCHMQ0aDr3ZtU0yrw\n",
       "U649MNJTcrBb/yAAAABdQZsOSeEOiZTAhn/+nhAAWzmVrdANmJgBJLm3zmBhOWHA/igDTn0s6/UI\n",
       "wcFbhp6o5yl2yc5f9zBFmv0YenbH6YC6zomQCsYJwZJ46EZlw2V2yT6ekbG6U3Kwd78wAAAAPkGf\n",
       "LEURPCf/ABiGfGKAAVJ8cgcsPXkPL2CvHgFs8fkKbVgDWD6HmXp5s1kVt1nHylj4IKTEu8hjNtr+\n",
       "lwpJAAAATAGfTWpCPwAc//UBXKcqFPHLIJNLuqb8+b+PHiBvJbXcr4qV1BuvnhlCzOR6HRsrsIi2\n",
       "pnF/6BrbOKO6WHx9mEAqDNmxkfS1JgVgYEEAAACPQZtRSahBaJlMCF///oywAFm5lScwjABLUnnp\n",
       "reCMkrvJ/nG/+QAtH/oDjYsXYuU0cz4uKF/CaehQo9xz1Y4KkieQvz38aw18qDBEMB0ct8CTjxCY\n",
       "BNFqEqm/8gUMMVuuZj2tpPbCEA0VvLrFiejaNdHjHVfQpmrPUjy14Z9McQjJiEMZeBQ0RO4g3hhU\n",
       "S2kAAABiQZ9vRREsJ/8AF+Z8YoABUnxyByw9W9ukOzeD+LZ4G9Ap5pOKeblrwCe9eEGXBqrEXnhf\n",
       "0doZk7TZYzjPr5UJupFfTzWYhWe+uHGfOIIGIt2pPtkcHJmhnysal7Eg36nDgWcAAABGAZ+QakI/\n",
       "ABxP9QFco9ISAgz6+4PFbMkZGDAdcqi3KSYLo/cTTuSzIR4hRorsIC3Vmy/4lrR2CgByGLcOCDkB\n",
       "5Hkge6SB/gAAAIlBm5RJqEFsmUwIX//+jLAAV3mVJzCMAEtSeemt4IySu8n+cb/5AC0f+gONixdi\n",
       "5TRxqNOvhtlCj3HPVlwpg8b2XBHAMgzjX6dcEsLbsKCiJcZnXZZg9L1QEBFkwoRQMBqnuY/CDiYG\n",
       "UVh0KPGMKgv5/n5ZSVc5n0xxCMmIQxl4FDRE7iDeGGEZuQAAAFtBn7JFFSwn/wAXSponKADRExiA\n",
       "FoM8TqnkCq+7z1etqGYFPGdxT6ctdfC2aUP1/70Al8QRiUQeEnaDLGOZ+uWbcU+T6gqy2Kzb1uoz\n",
       "txAI3/3amF5sDz8oKgW0AAAATgGf02pCPwAboncBXKPSEgIM+vuDxWzJGRgwHXKotykmC6P3E07k\n",
       "syEeIUaK7CAt1Zsv+Ja0dgoAchi3DggxgSGMZLefqAEd+8WasKrCXgAAAIBBm9ZJqEFsmUwUTC//\n",
       "/oywAFU5lSc1FQAWtuw3YfUV5qnhQl+IXuEwLUkPd809qNOvhI/DZ44UweN7LgjgGQZxr9OuCWFt\n",
       "2FBREuMzrsswel6oCAiyYUIoGA1T3MfhBxMDKKw7U/dH5II+wOCSrP3XUeGLajOPLq5I6zNTuxVy\n",
       "wQAAAFcBn/VqQj8AGwh8ZVgBnz21N2FXmUuTSA06K4hyybxqUhWb3zDo9qBB1s7r//SuBD94E5Gt\n",
       "AxFEjxJVurGv6Eb6eI5hGQfPOPRy/AxCVCPh9zK0MqxpDrgAAABJQZv3SeEKUmUwIX/+jLAAUmod\n",
       "huAFYm4jhdLF+Uhh3nv7CwCep214w9SaBzpSckxAe9yp1hO6fNLUcL24ybFP1szIvkV7lgd8IQAA\n",
       "AI5BmhlJ4Q6JlMFNEwv//oywAFKDRGTJ1gCtKvKS/fC23b6BYETh+vTp+s7LUux1R/AVACZYKmps\n",
       "XVUteLbhv7D7fQFmTqmdCkEbRk4Or0wo2gfPbRNM2RheX09+nJOcs4rkF0hbebZ5b1OR10EIfl89\n",
       "WByVNf4/xB5uq/MZVP3ugduqCqDWVqoIkRz9zYf3AAAASwGeOGpCPwAaaIClYzvkP0Wr7eltBgDx\n",
       "7WdV4azMMRzxBNf26oarSg1JgMSCkS5PUSaAuIQpHqchv2hkOyzQ8hXo8VnUdIbRPahNwAAAAGBB\n",
       "mjpJ4Q8mUwIX//6MsABQKh35tgCtJrsUxvoRo4jhdLF+Uhh3nv7CwCep214w9SaBzpSckxAe9yp1\n",
       "hO6fNLUcL25XrLZRzU/3gBF/Byy/M2kqf8l9SKutic68vBnDsWEAAABNQZpbSeEPJlMCF//+jLAA\n",
       "UCo19ZwBtM89Ia/P/3zsewR3nv7CwCep214w9SaBzpSckxAe9yp1hO6fNLUcL24ybFP1mXqZZEj4\n",
       "IcvUX2oAAABfQZp8SeEPJlMCGf/+nhAAT6WoDQC6gA8SrXhHCLKHevXjGcbozTqSVGXXu80IYs8m\n",
       "Ke8fNAocJxIhKPTiWiFAiQBeR5TY63AyW/O8QBlAOV/q00B7F3b+Ancxf4e4amEAAACEQZqfSeEP\n",
       "JlMCGf/+nhAATVUPH0wACdVh1eqwkV0xSBQIiZoXpF92LUET0zq4pAGoG2KTgOWdJd+8/Q8ICgni\n",
       "jSUY/Ul3+EhsqFSFjI7QfjEA3nn7Z3y3JRbt2TvF/JQ9b7j44zIjyhpGGbBx1brk4Gyw8lme+TQb\n",
       "Zxz+G5brLhUgWNbBAAAAXUGevUURPCf/ABUKmicoAK9A/MDEYM8ToFFddCNw/46lsM+dASTuKfnl\n",
       "rtoWzmh+zPehMviiMS0Dv67P6YyTP2azcun0XULWW7WcCt3Gd6IApKGSgCKwSyEf4lIVNwAAAFIB\n",
       "nt5qQj8AGSiApWOU9dRQAtpxVu6C0dka5rbLQY76f/6PCMXvbVnpYjbZ/dGnf06wCnIo+xqJFR+f\n",
       "hCfugQe8ctbFX3qTp0MNN63ipllnKUneAAAAkEGawkmoQWiZTAhn//6eEABLUfo+mAATqsOr1WEi\n",
       "umKQKBETNC9IvuxagiemdXFIA1A2xScByzpLv3n6HhAUE8UaSjH6ku/wkNlQqQsZHaD8YgG88/bO\n",
       "+W5KLduyd4v5KJMjtwqq0eUzjXNN3CmALUkupd2Tg4izPFme+TQbZxz+G5brLhUjlpVycTVTLcLU\n",
       "ZQAAAFpBnuBFESwn/wAUeponKACvpjfgzeRi4CLCAXLz+sb8DPnQEk7in55a7aFs5ofsz3oTL4oj\n",
       "EtA7+uz+mMkz9ms3Lp9F1C1lu1nArdxneiAKShitJ7CP88caakAAAABSAZ8BakI/ABiIgKVjlPWH\n",
       "aGyivmR0MS5nO7lbxu535lZ7bsWbOm7rVquZ9bKA0UKRZEJxkvzYrCkqp0MbpShSLPLyMZrk2QA0\n",
       "Eh1yoaE17jGQqQAAAINBmwVJqEFsmUwIZ//+nhAASVH6PpgADsd5VxigiBMUgUBgxvcjs7/6giem\n",
       "dVl5we95cQoYGLtrB4l6viBESoJ5s0kkP1Td/nAbART/41Sz7YtwOnPooE0itfBWU/XmudGqKkzT\n",
       "UX3wU9bEhYOSvfa2B9a/SnJjXns9nXpkpQQmDFyqkAAAAFtBnyNFFSwn/wAT6ponKACvpjfgzeRi\n",
       "4CLCAXLz+sb8DPnQEk7in55a7aFs5ofsz3oTL4ojEtA7+uz+mMkz9ms3Lp9F1C1lu1nArdxneiAK\n",
       "SheUn4qEU/zi+zUhAAAAUgGfRGpCPwAX6IClY5T1h2hsor5kdDEuZzu5W8bud+ZWe27Fmzpu61ar\n",
       "mfWygNFCkWRCcZL82KwpKqdDG6UoUizy8jYaZrMANPAdcqGiovwHtr0AAABsQZtHSahBbJlMFEwz\n",
       "//6eEABHUkg4AIg35xoRXesphbvOJm5m0ej1jBDj8EEiG/mT2VMGMQbJlzGL8yOSHmcGOJyy1w0v\n",
       "eBoey6asnwPI4+bzt0ffKuy1XB5tCYu1txLonQQMxZQe9pudyqCfAAAAQQGfZmpCPwAXQkRekAM5\n",
       "IZykbiNGNxdiXEbIkgrfRepYDNw4rpxytsIFAe+l9dF1VFBqtISEw9pb1QU1GfYhO4w9AAAA8UGb\n",
       "aknhClJlMCGf/p4QAEe+eyRIeAEKb840IrvWUwt3nEzczaPR6xiTHj0KHrlv4C/hH829UotI5drD\n",
       "uHeGDNk1O/ivkhbz2kQf6AoIKnTBI1xLlJit808LHonOSv/KiAs2yrlx+NV1liPc87Dp0yqbgTtj\n",
       "jgNrBikbrw96H9y9TMbYql/ILCG8F35n88JOfBvYTUSBF7taVYZKhA6vmtTkXJHd0iq+abJaFBmy\n",
       "MBDjiWjGssswraZsJCOKefx2u08yu06cU+sJxFcOl3YU09JMgEUH+h9l0Nq2ulVMq4fQK/HOVqd1\n",
       "LRtad9pEj4MTeSsAAABkQZ+IRTRMJ/8AE1iSPFvT+mwQAlZWFs2AuX6ckvKPLJk/OycHiWMdc011\n",
       "oFfBjS1Avd9dUHFSF0M27bA682NWaQolQNUxHWu6vUXk7fOht38Bxv6slzoPETNvwHB1cq5vJZwH\n",
       "+AAAAEgBn6lqQj8AFrtAtVgBUyR4ZlRSe2mZoB8W+4aIPZ1LvuCfrb/XNIlsu9xD63KOTi4L+iKA\n",
       "4eOdxRDZ7jPEk1RUITW4A997W7kAAAClQZutSahBaJlMCF///oywAEZ+hC4S4AE7b4A0IrvV8whm\n",
       "rihczSSwc5QX/a4oXKrG9PF2xXKyuaMRgnGjj0bYVADvanE2lIjzW2wtXw20JmIt7yI77xFjvfB/\n",
       "EI353elt4YD/cp8zD2CSpC4KO579iwIt56y1NF/8sTKKj5OdGqgQ9WfXVbYbLKbPcygfQKVEkIjZ\n",
       "CuwiEK0Y5AfhFespTuKbHXJmAAAAZUGfy0URLCf/ABLYkjxb0/psEAJWVhbNgLl+nJLyjyyZPzsn\n",
       "B4ljHXNNdaBc+pGIk7VaTEOMS64ak6E6v0h8o6kkJXlsxAEVm2wWq+aab99BDN9mHmwwSjXpmEJN\n",
       "/kHT7r8tbA/wAAAASQGf7GpCPwAWK0C1WAFTJHhmVFJ7aZmgHxb7hog9nUu+4O6cLmifpwVorWoW\n",
       "sx1Xs0yDmemAP/lvcgQ6f1V5M9b09E43/pYPjj8AAACeQZvwSahBbJlMCFf//jhAAQb5liLHQAfz\n",
       "aYgCMr6EbD5jcCQdE6/TNfT9O3nmV5afAizovweMd5yM6GWtwHFuRGj0dGT6qYsfb/cjCSzfh4BV\n",
       "iykCDnPMdtS6RBWV1UzbBj+875cw3FX6VTxLXKXZ7xabsdWSkINRP4F0+bVUs1XxSMEEWm9ayFq2\n",
       "wcOYFD7FuB0MQKfXFQLyMqJBOmEAAABiQZ4ORRUsJ/8AEliSPFvT+mwQAlZWFs2AuX6ckvKPLJk/\n",
       "OycHiWMdc011oGBVdfyipnqTfIeK6H7cRCKTm/op4kCClIxX9Dx/AkzAEZa7CV72j0GJ0FOTLUTf\n",
       "4/2/18hBgd0AAABEAZ4vakI/ABWbQLVYAVMkeGZUUntpmaAfFvuGiD2dS77gz4zvwcN5Jfho5KIV\n",
       "gz+60oMmVL3NbuOt4Y3UgY4AlsFp2YAAAACdQZoxSahBbJlMCF///oywAEJ+hBvN0ALn3AUIvCRX\n",
       "NTrbkSwGhdw1nuBfQ8pHrl/9ZZNF2xWj9tA9erHXTu2usKd53sXiERAWK3nx2R8IWsutvoOt1OjD\n",
       "C9C//Qr793r/FI9HE/SZzXRgs8Gbb1kEGISEtMYnGXmHZ5SS4gayCGse/Xu3YUOUFE1sXuHU6H6z\n",
       "nPjHcxJl3cMWy/8tgAAAAGRBmlJJ4QpSZTAhf/6MsABAAXYYMAE6pqCrpcpeMN8Vz39hYBPU7a8Y\n",
       "esDg60d1esorcYSRTynIfi9R5J5OyrN6t+YOXOAnrdn+sAzJXyYfvfLCOsGpm01+xa67V53YffXM\n",
       "xWpBAAAAbkGac0nhDomUwIX//oywAEABYhgwAQ/j8hXPu9ym/Hy9NSMIO0QLLEGQU0vqu0IUvoII\n",
       "fEojBrx4MJzHxmYMIbPw3Mv0WFED/ZWJQEp0+mpssNO0dxoK3LPdttN7EgVrgIkTnxVfe9jdxNfy\n",
       "fRz4AAAAgUGalUnhDyZTBRE8L//+jLAAQDPbnikvAAnbfAUbD8PspYXcdx/6n9y+LAmu2DcA7Bps\n",
       "Ct9Be98wzVDMaf2Yeem08rHkSaU9+6ysT/OQ/QZ3lwzsYWX4dVBi8NfPYkfe+zeN+KX+dXDidMGR\n",
       "QFVGXu9Y35j77qN5coZ0OxCwvb0c+AAAAFQBnrRqQj8AFQUlXXwACBPsUXWb4PCZhEKYDa80ZDHE\n",
       "fOi1HCd7IreHHNK7CUw9WHLO+phQlWqEXIGyFhrp3DeTA2H4TJO3qtELvsLkSl25eKFYVsEAAACH\n",
       "QZq3SeEPJlMFPC///oywAD68J0yzAxTOADipr+udP9zjFR2c3AbrZWl5xnkcoiGnctsugULVDsUC\n",
       "BFDdYhEn8LbtX/ev/bdv5yShzYRKoA9Iuo1A1UnDBVTIzgK7YQdGIVzlypa1NKPx9m9QHf86L3d4\n",
       "+MzrWXO7z+q+R+A3Wq5cImhQE82gAAAASwGe1mpCPwAUclCH+RGXFGekLojXxZSTbB4Ct//gS4NI\n",
       "zfozZ33dj9Vz/rhwGiBSNIhQMl6bFoUmFOijdQUKpZ6eRvU/fFwJSTU1IQAAAFVBmthJ4Q8mUwIX\n",
       "//6MsAA8/INigAO/tWnae2Hum9Vmy1OqkokQMzr2Ajj8A0oCTjKYBa3j8dSRC1yylQ7iJeftvH0+\n",
       "TQyNQClHhIF+KFgocMWSz5tBAAAAT0Ga+knhDyZTBRE8L//+jLAAPVwnRPLJ9qc5JfTWYxw1qAtt\n",
       "1olKoxK0cThIBWiezr8nvJvB8DZpKKnl9L2Z+F6kABMlA/sSggJYz98zzuAAAABLAZ8ZakI/ABPi\n",
       "UIf5EZcUZ6QuiNfFlJNsHgK3/+BLg0jN+jNnfd2P1XP+uHAaIFI0iFAyXpsWhSYU6KN1BQqlnp5G\n",
       "9T98bCcJNTUhAAAANUGbG0nhDyZTAhf//oywADun4dqYa9GgRioYs5veeRrmZRdKl6zIqSINi+54\n",
       "LZ4ABnO49AfTAAAASkGbPUnhDyZTBRE8L//+jLAAO/wnP227t7r6zRIJnlnyyUh6xw76z1mBtBS4\n",
       "Gd/brfDSen4Sj2p7yqeuyvoBAEvinR9kHK9Hv0CBAAAASwGfXGpCPwATVfg2uvsLijPSF0Rr4spJ\n",
       "tg8BW//wJcGkZv0Zs77ux+q5/1w4DRApGkQoGS9Ni0KTCnRRuoKFUs9PI3qgU4vxOEmpqQAAADBB\n",
       "m15J4Q8mUwIX//6MsAA6R+HamGvRoEYqGLOb3nka5mUXSpesyKkiDYvueCxq+6AAAABJQZtgSeEP\n",
       "JlMFETwr//44QADjb/zgBIyCOF32mZTPbv+fRe28V4qdr3pCu6yRE7I46+bSyN02Ld/yACALODwn\n",
       "8OFKMG6ziYWmgAAAAEsBn59qQj8AEtX4Nrr7C4oz0hdEa+LKSbYPAVv/8CXBpGb9GbO+7sfquf9c\n",
       "OA0QKRpEKBkvTYtCkwp0UbqChVLPTyN6n75IE4SampEAAABUQZuBSeEPJlMCFf/+OEAA3MuCLdTA\n",
       "QY7DxqkMjAd3LgcfJ++wn3QN7aO6TqC/NiNDe1o0r6s+cq3dxEGk+oa9yiSev+lpvZ81LW9gDPFO\n",
       "w857SIXoAAAAREGboknhDyZTAhX//jhAANyeQF1SKLqjjF/8k8YcGC6bI+fRUv880O1+mWoDVk4w\n",
       "haUWw9EG9UR/WysWuOaOWevVXQmRAAAAXEGbw0nhDyZTAhX//jhAAN3v/Nd0GkAFBuUuv6Ne+JAA\n",
       "XQ7loxKZ5QW+g47vxMn8WOVtisa7N7s4pdPNmryiWQHcm87Dg4cOx97JK6oUIC3UE9JTUM5L3dVD\n",
       "SelQAAAAL0Gb5EnhDyZTAhX//jhAANfLaeXUwEGOw8apDIwHZYNy3JbXXOvPyb2vpeu+kdpnAAAA\n",
       "REGaBUnhDyZTAhX//jhAANeeQF1SKLqjjF/8k8YcGC6bI+fRUv880O1+mWoDVk4whaUWw9EG9UR/\n",
       "WysWuOaOWevVXQoxAAAAWUGaJknhDyZTAhX//jhAANjv/Nd0GkAFBuUuv6Ne+JAAXQ7loxKZ5QW+\n",
       "g47vxMn8WOVtZuflJ+ZcrLFDvgyw5egax+2JyuI+dEAWrq5oqDAKO8Ihv5iy4j3RAAAAL0GaR0nh\n",
       "DyZTAhX//jhAANLLaeXUwEGOw8apDIwHZYNy3JbXXOvPyb2vpeu+kdp3AAAAREGaaEnhDyZTAhX/\n",
       "/jhAANKeQF1SKLqjjF/8k8YcGC6bI+fRUv880O1+mWoDVk4whaUWw9EG9UR/WysWuOaOWevVXQrQ\n",
       "AAAAXEGaiUnhDyZTAhX//jhAANPv/OtYTJoAzqer8ExU9g2gJnLvP3/Ckj/u2DciVs7wY6iS3DZd\n",
       "nRKOhE53X/qLcClCCk1A2aiaINAcNDj9Py3HBICntSKl7uqhpPVIAAAAT0GaqknhDyZTAhf//oyw\n",
       "ADUUf6jbgDmCV2L2PjDfFc9/YWAT1O2vGHqTQOdKTkmID3uVOsJ3T5pajhe3K9bPvdRuS2JoKq+V\n",
       "Xd3Mai8aJeEAAABlQZrLSeEPJlMCF//+jLAANQsazggGNN64CPfBH6VIoXcnKNMgh6m7tW4gRgsh\n",
       "IFgLBwJ5QABdu+zjf37CjE1hsba4khHAXDj9AiBW8oiaJ0AktGHkRdwNjaQC0HvvMUvOEQDzqvAA\n",
       "AABpQZrsSeEPJlMCF//+jLAANQPc/1SY5O1gBbekLL76ySGqtgVvmb3vemmayTDgek99a5wrOvus\n",
       "rE/zkP0VXMwGh9uHMXmxKVNkPumg7HXUqDVqhto1ZZELcgi12aEmrG+JG+UqL/n6iUXAAAAAV0Gb\n",
       "DUnhDyZTAhn//p4QADNrvdJKAC+Vive9iqvp5baYH5zQa8ZbevTE2TLmMX6AB5HdWJ7HACoXxwN9\n",
       "p92RW/k/rWN0EidwyXKVM6f3pi3Lkz/i1LsdjwAAAJZBmy9J4Q8mUwURPDP//p4QADN28I+mAAOx\n",
       "0rgxk+pMKtxr6lPj2xeWEBnzvmB7Oq2cgXciWBjGLtrC/n9PDPKIlXPebPxQqidF46Avczf0ckmk\n",
       "KbnXk5l3RGAqLza17AHeTjD9TFE8/55yUBDQSk7Jzsh9dFgh0/RpkO0fxe0qddwvWv0pqRoX3qnj\n",
       "YUtfsHwyFYCp/mEAAABbAZ9OakI/ABBV77awAqakLyDrweTtJka6FjOkNXo53XzzEaSNNdw5BJC8\n",
       "rfXRg1tKQ43NNQ2e7fxj5IYbmXccJvrnhlSPFC21YvjpbO18dMv7TsVzoDdo6kED0wAAAHdBm1FJ\n",
       "4Q8mUwU8M//+nhAAMjTrHACFN+VcX13rKYW7WHVlsbUB7EvmXSPDNoEINBndVjIjfeinXSJgsaIs\n",
       "F/slJBbNVe18ZfJ2Y2NOFid5nusCOXM5ZvzA345vXZ5LXXk0UHRafa2yYOSX2BpTWfXPA9YsnG4d\n",
       "0AAAAEsBn3BqQj8AD9f++v7gMe+EALVt5BIGPqfe2pyvZn2PWVAJRSqK0AdqwS0pjKeH+YmTzchj\n",
       "pTrJsNcgXNyfOIDBJNw7aoONbpYYpIAAAABlQZt0SeEPJlMCGf/+nhAAMjbvgXWADahgtCvrvWUw\n",
       "t2sOrLY2oD2TJPIOc0CZwmxD3tNyRHNuCn1VM9BK1yO+6MjbLJn0/oHWynSsxg644MhnG2tCfIn5\n",
       "UStWkbuOdK6Crjf39VUAAABVQZ+SRRE8J/8ADYCcHi3qkKxoAEc+acwPq0IHfCKhXIxgOzZoQCFV\n",
       "OkrnSgYd2ERoSSTycqifFlo3V2XoMFm5piOPhvpiXQ1D8vpg5trX8fvjy40CggAAAEUBn7NqQj8A\n",
       "D3/++vIJeEALVt5BIGPqfhhlONT30grOoO8pbqo0/xWHRQgvjJJ5fsfLBuYJV23GY92Q60qGyRwf\n",
       "Z5StR8AAAACoQZu3SahBaJlMCGf//p4QADE/BpTJIARBvyri+u9ZTC3aw6stjagPZJjxQf9Ly1iy\n",
       "iGUd9QX+236Y4osjC7Ub6BGToIbA+yao01AONfF5rEnB4eWQ/MUNUK65+eVag0KAInREwpRsryPG\n",
       "6OYjSeWTf+Y0T/24jMnlLNsdlgLcFGEXf2Or5A5Qtz5Ou6q71qJAnRjySIKCEHcsHzol114XRfan\n",
       "HprL+0HpAAAAWkGf1UURLCf/AA0wnB4t6pCsaABHPmnMD6tCB3wioVyMYDs2aEAhVTpK6IoOxnX+\n",
       "H8ZY1ytoRh968wcKhdxDfc80YRxwfWUa8kyM4VDYMzn0l0st0Fgvnp0D5gAAAEcBn/ZqQj8ADyf+\n",
       "+vIJeEALVt5BIGPqfhhlONT30grOoO8pcSpOkoLga2OcrFE3eo/5zUmtMITV3IeSm3e/mX0BVwZc\n",
       "goA2YQAAAG9Bm/pJqEFsmUwIZ//+nhAAL/8GlMkgBEG/KuL671lMLdrDqy2NqA9kmPHoUPW/i4pQ\n",
       "j+beqUWkcu1h3DvDBmyanl92BZtlrYKqjFokHl32msP5/K3icl5NqxW3slId5j+jXIsblXkYmVnk\n",
       "kze4g4EAAABYQZ4YRRUsJ/8ADOCcHi3qkKxoAEc+acwPq0IHfCKhXIxgOzZoQCFVOkrpyg7GemVH\n",
       "is3xfRrxULllrQJu4otTiMJlkR3XQKekHwSWnyg58oIJgfL4ZWBdwAAAAEwBnjlqQj8ADs/++vIJ\n",
       "eEALVt5BIGPqfhhlONT30grOoO8pc6pOkoK/r6OcrFE3eo/5zUmtMITV3IeSm3e/mX0BVwlufK1U\n",
       "RrvFAF3BAAAAZkGaPUmoQWyZTAhf//6MsAAu4aJeeQAbULcfGJD1fMIZc0FlguYbQjTz8l5jHe+y\n",
       "1JAAliu+63dParFrrzhTZaItXJ+buOV5Rhu9JYu04PONIUjMokolzvcOLVytOdJhQ+XwoJkQ8AAA\n",
       "AF1BnltFFSwn/wAMkJweLeqQrGgARz5pzA+rQgd8IqFcjGA7NmhAIVU6SusQrzLX8FhTW+L6NeKh\n",
       "cstaBN3FFqcRhMsiO66BT0g+CS0+UHPlBMYZvod5X0MZh5SeBZUAAABLAZ58akI/AA5Rt7d0Bn0A\n",
       "Nxt5BIGPqfiZlBVf20y0tpfcoC7j57gquyQZ2MWjfSjjn5TK1gh5XQRy8c5OiXKJJXCaGVfDMjLl\n",
       "wA1JAAAAfUGaf0moQWyZTBRML//+jLAALcGiXnkAH7br8NDvPV8whlzWV9ILpj/kaepYqOtraMw3\n",
       "1lq8vrwfELwzWDT9EKU/mvLw5rremfuSl9PSOSLjBc4o+K1BD41WkYlcdj9NiuPpGOVsf0S0h8Vq\n",
       "EEBYYOLWVyiw1q1qx5MyUkTgAAAAVgGenmpCPwAOf/GulObkXCABqRea5TJ7sbZsm0scfiPAWQx9\n",
       "saMK9gvSzcuOeE0iE1Y9PMKuvKtEN2BLXJn3JyKS+aP108+H2t0UXP3fcna+nqHAUMCAAAAAjEGa\n",
       "gUnhClJlMFLC//6MsAAsn1gj/gAXPuAo4ax2TdKtwL6d9E+yzdPXQOKCnXgbwCEUuLmqnZLxQ2LL\n",
       "twmJH3zD1MlNLHKagWdDJTM6tDE/IQ82h4VmKff9iRuU4mxx3xha4nebFeFZychby4zc3bVyo9BU\n",
       "qTKEVSsA6Z75MOfqgnf/b3i+lZUEnQfBAAAAUwGeoGpCPwAOK4Fu1AZS4oikhbEa+UKSiu9yKw3c\n",
       "38y5kd4rM/2YGfxElhzBR3M+fbU9YphuXf61ItGoiuoUZ3w3Hs2gIzUUAbSLgAG6zppDjBgQAAAA\n",
       "XUGaoknhDomUwIX//oywACuJ9cgny4ASIIfrt3DGiu7ttxVs7mxuEXOcL2s7PjOFruZ0x9J3PlRm\n",
       "eLUwVCmOvHT8LnE/2OlENuNEQa2Mp+LpMW+jaV4jTUhBCspW0QAAAQJBmsRJ4Q8mUwUVPC///oyw\n",
       "ACuBoivedYAW3uAo4ax2TdKtwL6d9E+yzdPWzpxjK1/NQCEWQ+/2S8UNiy7cJiR98w9TJTSxymoX\n",
       "nMlDT84jaj4nL7u6PktcYmPRwPWI/amK27U9V4mowMHEdgTCK4GHqb8aAs8bpaHfHJMVyRoKKWB7\n",
       "6IT8COfqgyeTdYsthkHxnREb0pxzyFK3XgPqV2U8w1eE1tds7eH5CqDkIFL+8gE/CoGhvFFyky/v\n",
       "RPjsetp+bOHpRqdE+W46ZxvZ9Tk2949vOO8kQK0ZM2OI+JqSAlG6Ax3cBtvCR56SYNgsaRhUzvBU\n",
       "4co3JwyKKxMcuY+6DxgAAABiAZ7jakI/AA3UPjKsAKmpC8zzKWFaWulFpXERNSeq4rbP98xiPalQ\n",
       "dbpqgnclopainFKr6GhVSW8Q3mZ4bgzHGb6PX00/98NsfPRrtzCzQEalEzgvuVdIUdfqXNFkaYsz\n",
       "QUkAAAB5QZrlSeEPJlMCF//+jLAAKmniDvRbYATV//mC8P12uI2Vev9vaF5heo1/rYhKhMMiqx5p\n",
       "MTYSS6EJKyF9F+wYWEzMm7fDohCY3YjPNT5DatAReelQvRb/4+HeKzT7C9zRJI52b7+wpWUXpOBo\n",
       "UgAD9FEz5l4+DcpHdQAAAFlBmwZJ4Q8mUwIX//6MsAAqXn+qywARB/+9Ky+7PQbonqyFvRI7q8KZ\n",
       "2YCzpiEr1tfq0lUvVCZvxJeVGW+S32LrO+pM1V+6FBsFEORFT6lZR12qbcl6EEy7gQAAAIFBmyhJ\n",
       "4Q8mUwURPC///oywACpysdxY4AFz7gNiCQFRz/KEMuayvpBdMf8mgAm6NtsbQnh6iyxfPE1k8dVD\n",
       "iCOpQ74wC8CPvTmkSxmld8y067uSXaKAFM9+vskSeFVQaYl8gpy5nLPTgVZO2VYq3WDaGbYtxOqo\n",
       "77blH+aseLMic+8AAABjAZ9HakI/AA2BMq6+AAP8Dipu0Hv8lhm3xHZOdTijCw4VVc6sWtiaTGFA\n",
       "5wSqy62M/EuOBMdVgS49tm7l7LXCZGv+/6s0BjNSP+h6LIQqO/j2UYgi8isL/jSNVageQ+hM64Ju\n",
       "AAAAgEGbS0nhDyZTAhX//jhAAJ/yfV8QYgCOtendQetua3BqSB5eb0xMLZxRbVqdg+G6ZxgUV3ts\n",
       "CQv9V5UOVByW0jjUveJokifK2MMNIZ6EZz/iL2WedZm2/vZ4cPlPn08NawF58qG2X+ple1m/ErSP\n",
       "pf3gna0XuyrIUb6TeWQFQotoAAAAa0GfaUURPCf/AAsVTQ1k+kOOUzi4WfPCZj0EzFHgSyhHwr2p\n",
       "6tKc0/gIwD0Q1IULrIckA8GQQGrqrMhHNTOJk+i3T8Dam9pXLxicxBJorZraT8uv4g8BmYA+Nvqa\n",
       "ABN7pl/2o9o8DGrpmCpfAAAAUwGfimpCPwANMTKuvgAECfbAuglwXYqFMN7vnRrEGhqbf/a8R/zc\n",
       "wDwGGhaEbo+q2mHKheEH2Zy2Ei9QLtGV3tB7i9STJnB//PwCX4ZFzilxmBxwAAAANEGbjEmoQWiZ\n",
       "TAhX//44QACaef5sppTFp0OUSFoeIXSV5DMtUxftXA5XF/iDTlf7soRXvMsAAABaQZutSeEKUmUw\n",
       "IV/+OEAAm36TUb/bQ98qg1mEm731tx7x8Nq4gRAtHPtXWsFmEJ21qRjZ1Bc1EgpwMtcasM3VRoDD\n",
       "+D3LCUIvyoylH+r3MTPjH53XTicpr1lRAAAAO0GbzknhDomUwIV//jhAAJab+eXUwEGOw8apDIwH\n",
       "fB+SvO8OKSTtdeG1M2BXU8jfaPemqnMXwqF1+zErAAAAL0Gb70nhDyZTAhX//jhAAJZ5/lL96bP1\n",
       "GnTDUZI8a6xrhgQQCbI4nT9jTCa50NNDAAAAVkGaEEnhDyZTAhX//jhAAJd0TVavZtAE7uTd7624\n",
       "+JAAXQ7loxKZ5QW+g47vxMn8WOVtZuflJ+ZcrLFDvgyw5egax+2JyuI+dEAWrq5ibSU1DOTlNes+\n",
       "AAAAL0GaMUnhDyZTAhX//jhAAJKb+eXUwEGOw8apDIwHZYNy3JbXXOvPyb2vpeu+kdt+AAAAL0Ga\n",
       "UknhDyZTAhX//jhAAJJ5/lL96bP1GnTDUZI8a6xrhgQQCbI4nT9jTCa50NNNAAAAVkGac0nhDyZT\n",
       "AhX//jhAAJN0TVavZtAE7uTd7624+JAAXQ7loxKZ5QW+g47vxMn8WOVtZuflJ+ZcrLFDvgyw5ega\n",
       "x+2JyuI+dEAWrq5ibSU1DOTlNetSAAAAL0GalEnhDyZTAhX//jhAAI6b+eXUwEGOw8apDIwHZYNy\n",
       "3JbXXOvPyb2vpeu+kduWAAAAL0GatUnhDyZTAhX//jhAAI55/lL96bP1GnTDUZI8a6xrhgQQCbI4\n",
       "nT9jTCa50NNVAAAAVkGa1knhDyZTAhX//jhAAI90TVavZtAE7uTd7624+JAAXQ7loxKZ5QW+g47v\n",
       "xMn8WOVtZuflJ+ZcrLFDvgyw5egax+2JyuI+dEAWrq5ibSU1DOTlNetmAAAAOUGa90nhDyZTAhX/\n",
       "/jhAAIqb+eXUwEGOw8apDIwHfW7VYEOSCl8rZdOfeblwgcowTFa2w99/Uow3VQAAAD9BmxhJ4Q8m\n",
       "UwIV//44QACKef5S/emz9Rp0w1GSPIGWRH1+3NYCbmCGVLzy1jyRM5SJHKkEDgZNZguKl+OvRtcA\n",
       "AABZQZs5SeEPJlMCFf/+OEAAi3RNVq9m0APx/m7hOcPXxm+VvJEjEpnlBb6Dju/EyfxY5XhsAvw2\n",
       "OS1Fl/9Pu9P5+jDqtrer4Elw7uMmwUFrCg+kpqPMrPa9b0AAAABIQZtaSeEPJlMCFf/+OEAAhnn/\n",
       "RiAIjyUimHzA61z39hW/fU7a7gftAOgR9NDyK8QhlAwaI2Sp3Aohe38NlDwiA29VFQfrPFD5AAAA\n",
       "Q0Gbe0nhDyZTAhP//fEAAU/21Ha9wdaqjC27ICop0HfGTxnr30LsARvQc16knktwtcNGOJSSg0Zm\n",
       "gU7o54y5osN0DpAAAABDQZucSeEPJlMCEf/94QACDe6eUOfQa2JJc/w3pEeG6w8l77/UR2WhgBnM\n",
       "3f2EgwWun3rWh9LgHcXEefBmGl03PahPyQAAB4dtb292AAAAbG12aGQAAAAAAAAAAAAAAAAAAAPo\n",
       "AAAMNQABAAABAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAEAAAAAA\n",
       "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAGsXRyYWsAAABcdGtoZAAAAAMAAAAAAAAAAAAA\n",
       "AAEAAAAAAAAMNQAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAA\n",
       "AEAAAAADYAAAAfgAAAAAACRlZHRzAAAAHGVsc3QAAAAAAAAAAQAADDUAAAIAAAEAAAAABiltZGlh\n",
       "AAAAIG1kaGQAAAAAAAAAAAAAAAAAACgAAAB9AFXEAAAAAAAtaGRscgAAAAAAAAAAdmlkZQAAAAAA\n",
       "AAAAAAAAAFZpZGVvSGFuZGxlcgAAAAXUbWluZgAAABR2bWhkAAAAAQAAAAAAAAAAAAAAJGRpbmYA\n",
       "AAAcZHJlZgAAAAAAAAABAAAADHVybCAAAAABAAAFlHN0YmwAAAC4c3RzZAAAAAAAAAABAAAAqGF2\n",
       "YzEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAADYAH4AEgAAABIAAAAAAAAAAEAAAAAAAAAAAAAAAAA\n",
       "AAAAAAAAAAAAAAAAAAAAAAAAAAAY//8AAAA2YXZjQwFkAB//4QAZZ2QAH6zZQNgQflhAAAADAEAA\n",
       "ABQDxgxlgAEABmjr48siwP34+AAAAAAcdXVpZGtoQPJfJE/FujmlG88DI/MAAAAAAAAAGHN0dHMA\n",
       "AAAAAAAAAQAAAH0AAAEAAAAAFHN0c3MAAAAAAAAAAQAAAAEAAAJwY3R0cwAAAAAAAABMAAAAAQAA\n",
       "AgAAAAABAAADAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAE\n",
       "AAAAAAIAAAEAAAAAAgAAAgAAAAABAAAEAAAAAAIAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAQA\n",
       "AAAAAgAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAAAgAAAAABAAADAAAAAAEAAAEAAAAAAwAAAgAA\n",
       "AAABAAAEAAAAAAIAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAADAAAA\n",
       "AAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAEAAAAAAIAAAEAAAAA\n",
       "AwAAAgAAAAABAAADAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAIAAAAAAQAAAwAAAAAB\n",
       "AAABAAAAAAEAAAIAAAAAAQAAAwAAAAABAAABAAAAAAEAAAIAAAAAAQAAAwAAAAABAAABAAAAAA0A\n",
       "AAIAAAAAAQAAAwAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAA\n",
       "BAAAAAACAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAAAwAAAAABAAAB\n",
       "AAAAAAEAAAMAAAAAAQAAAQAAAAABAAACAAAAAAEAAAMAAAAAAQAAAQAAAAACAAACAAAAAAEAAAMA\n",
       "AAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAEQAAAgAAAAAcc3RzYwAAAAAAAAABAAAAAQAAAH0A\n",
       "AAABAAACCHN0c3oAAAAAAAAAAAAAAH0AABDGAAADNAAAAKEAAAIGAAAAswAAAKcAAABhAAAAlwAA\n",
       "AG0AAABWAAAAVwAAAE0AAABhAAAAQgAAAFAAAACTAAAAZgAAAEoAAACNAAAAXwAAAFIAAACEAAAA\n",
       "WwAAAE0AAACSAAAATwAAAGQAAABRAAAAYwAAAIgAAABhAAAAVgAAAJQAAABeAAAAVgAAAIcAAABf\n",
       "AAAAVgAAAHAAAABFAAAA9QAAAGgAAABMAAAAqQAAAGkAAABNAAAAogAAAGYAAABIAAAAoQAAAGgA\n",
       "AAByAAAAhQAAAFgAAACLAAAATwAAAFkAAABTAAAATwAAADkAAABOAAAATwAAADQAAABNAAAATwAA\n",
       "AFgAAABIAAAAYAAAADMAAABIAAAAXQAAADMAAABIAAAAYAAAAFMAAABpAAAAbQAAAFsAAACaAAAA\n",
       "XwAAAHsAAABPAAAAaQAAAFkAAABJAAAArAAAAF4AAABLAAAAcwAAAFwAAABQAAAAagAAAGEAAABP\n",
       "AAAAgQAAAFoAAACQAAAAVwAAAGEAAAEGAAAAZgAAAH0AAABdAAAAhQAAAGcAAACEAAAAbwAAAFcA\n",
       "AAA4AAAAXgAAAD8AAAAzAAAAWgAAADMAAAAzAAAAWgAAADMAAAAzAAAAWgAAAD0AAABDAAAAXQAA\n",
       "AEwAAABHAAAARwAAABRzdGNvAAAAAAAAAAEAAAAwAAAAYnVkdGEAAABabWV0YQAAAAAAAAAhaGRs\n",
       "cgAAAAAAAAAAbWRpcmFwcGwAAAAAAAAAAAAAAAAtaWxzdAAAACWpdG9vAAAAHWRhdGEAAAABAAAA\n",
       "AExhdmY1OC40NS4xMDA=\n",
       "\">\n",
       "  Your browser does not support the video tag.\n",
       "</video>"
      ]
     },
     "metadata": {},
     "execution_count": 18
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": [
       "<Figure size 864x504 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAt4AAAHBCAYAAABe/LyMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoZUlEQVR4nO3deXxddZ3/8fdN0qRNSxtoKZStRaQqyjaiDAgoaytQhQq4oCgoi5SfLI4LjiiCIjAqRQcGcWEEERmxHSlqqywDCgMOiEUssrcFCpQWQilpkya5vz8YM2JLm9Tkm6Y8n4+Hj0dycr7nfq45j+urx3NvKtVqtRoAAKBP1fT3AAAA8GogvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAqo6+8BuuP/feoL+da/nNXfY7COmTdvXn+PwDpo7Nix/T0C6yCvF6yK14tXsTNH/NXXzxd72AFxxbv5+SX9PQIAAPxdBkR4AwDAQCe8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABRQP70fnPZYTTv1c3n7QETnkA8fmpt/ctsY1Dy5enqOvmZtr5zSntb2zwJQAANC7ioZ3e3tHPvnPZ2eP3d6SG6+9Kv/8TyfljK98PfMee2K16+prK2nr6MxFdyzMGdcvEN8AAAw4RcN77vzH8syiZ3Pk4YektrY2b/mHHbPjm7bLL3514xpWVjKsoTZjm+oz+8mWzHpgSZF5AQCgt/T7Pd7VajUPPzqvW/tWKpWMaqzL9DnNfTsUAAD0srqSDzZuqy2y0YYjcvmPf5ojDz8kd959T34/+97ssvP2K+07bcbMTJ8xM0nS2tbWtb2xviYLlqwoNjMAAPSGouFdV1eXr3358/mXb347l1/107zhda/N/u/YI4PqB6207+RJEzN50sQkyV5HnNi1vaWtM6OHrrw/AACsy4qGd5Jsu83WufTCc7u+P2bKP+WgCft2a221Ws2ilvZM2XV0X40HAAB9ovg93g8+/GhaW9uyfPnyXPHjaVm0+LlMmrjfatdUU83S1o7Ma27LjmMaM2H88ELTAgBA7yh+xfsXv7op//nzWWlv78jOO7wxF33t7NSv4laTv7aio5r62ppM2XV0Jowfnoa6fn9PKAAA9Ejx8D7548fk5I8f06M1244cnMsOG9c3AwEAQAEuHQMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABRQV/oBFzz5dM6denH++Kc/p37QoOzz9rflkycdl7q62ldc8+Di5Tn6mrk5dLumTBg/PA11/r0AAMDAUrxgz516cTZqasrMn16RK7/7rfx+9r255mc/X+2a+tpK2jo6c9EdC3PG9QvS2t5ZaFoAAOgdxcN7wZNPZ7+990hDQ31Gjdwwu7/1zXl47vw1rKpkWENtxjbVZ/aTLZn1wJIiswIAQG8pHt7ve8+78qsbb8ny5cuz8JlFufWOO7P7W/+hW2srlUpGNdZl+pzmvh0SAAB6WfF7vN+80/b5z5/PytsPPCIdnZ05eMK+ecceu62037QZMzN9xswkSWtbW9f2xvqaLFiyoti8AADQG4pe8e7s7MxJn/pC9t5z9/xm5k9z/c9+lCVLl+ab375spX0nT5qYKy6dmisunZqG+vqu7S1tnRk9dFDJsQEA4O9WNLyXLHkhTy98Ju899ODU1w9K04jhedfE/XLr7Xd2a321Ws2ilvYcul1T3w4KAAC9rGh4NzWNyOZjNsk1P/tF2ts78sILS3PdrBsy/rVbr3ZdNdUsbe3IvOa27DimMRPGDy80MQAA9I7i93iff9Y/5+v/eml+cNU1qampzS47b5/Tphy72jUrOqqpr63JlF1H+xxvAAAGpOLh/bptX5NLLzy3R2u2HTk4lx02rm8GAgCAAlw6BgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAXUlXywPSce9rLvW9vacti7D8ynTz5hteseXLw8R18zN4du15QJ44enoc6/FwAAGFiKhvdvZl7T9fWyZctzwKEfzH7v2GON6+prK2nr6MxFdyzMb+cvzdn7bSa+AQAYUPqtXm+4+dZstOGI7LzDG7uxdyXDGmoztqk+s59syawHlvT5fAAA0Jv6Lbyvm3VDDjxgn1QqlW6vqVQqGdVYl+lzmvtuMAAA6ANFbzX5i6eeXpjfz743Z3z6E6+4z7QZMzN9xswkL90L/heN9TVZsGRFn88IAAC9qV/C+7pZN2an7bfL5mM2fcV9Jk+amMmTJiZJ9jrixK7tLW2dGT10UJ/PCAAAvalfbjX5xa9uzEET9unxumq1mkUt7Tl0u6beHwoAAPpQ8Sves++9LwsXLe7Wp5n8RTXVLG3tyKKW9uw4pjETxg/vwwkBAKD3FQ/v62bdkL333D1DGxu7vWZFRzX1tTWZsuton+MNAMCAVDy8//mTJ/V4zbYjB+eyw8b1/jAAAFCIS8cAAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAF1PXHg8664eZ85wdX5amFz2TkRhvmzM+ekp13eNMr7r/ghRU584YFBSdkIGhufrG/Rxhw2jurmbekIw83d2aToTUZNaQmE8bVZ68t69NQW+nv8QBgvVY8vG+/8+5869J/z1e/8Jm88Q3js2jxs6VHgFel9s5qbl+wIs8sq2ZwXbJpY01a2pPL/7Q8dz61Iqe9Zaj4BoA+VPxWk0svuzIfO+r92f6Nr09NTU1GbzwqozceVXoMeNWZt6QjzyyrZoP6pL62kkqlkqGDKtl8WE3mLO7ILY+19feIALBeKxreHR0dmXP/Q2lufj6HfODYHHjYh3Pe1H/L8tbWkmPAq9LDzZ0ZXJdUKi+/ql2pVLLh4JrMmiu8AaAvFb3V5NnnmtPe3p4bbr413/3Weamrrc1pn/9yvnfF1ZnysaNetu+0GTMzfcbMJMny5ctLjgnrpZb2aoYNWvXPhtQlC1s6yw4EAK8yRa94NzQ0JEneO3lSRo3cKE1NI3Lk4Yfk1tvvXGnfyZMm5opLp+aKS6dm8ODBJceE9VJjXSUrXqGtl7UnI4f4kCMA6EtF/5d2+AbDssnGoxLv34LitmmqyfL2pFqtvmx7tVrNc8s7M2FcfT9NBgCvDsUvcU165375j2nX5dnnmrPkhaW56pqfZc/d3lJ6DHjVGTu8NhsPqeSFtqSto5rOajUvrqjmiaWd2W5kbfbaUngDQF8q/nGCHzvqfWl+fkkmf/D4NNQPyn5775ljPvje1a7ZbINBOXPfzQpNyEAxb96K/h5hwGntqOaWx9oya25bFrZ0ZuSQmhz1xsE+xxsACige3nV1dfnsqSfms6eeWPqh4VWvobaS/cc1ZP9xDf09CgC86ng3FQAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AACigric7PzrvsTw677GM2WR03vC61+bPDzyUf/3OD/Jc85Lsvuub8/FjPpiamtW3/HEnfzb3zrk/tbW1SZKNNx6ZaVd8e7VrHly8PEdfMzeHbteUCeOHp6HOvxcAABhYehTel/77j3LDzbfmn/7fcRn/2q1z2ufOzqJnn0u1Ws2DDz+aIYMH55gPHrHG43z65BNyyMETuv249bWVtHV05qI7Fua385fm7P02E98AAAwoParX++5/KEmy6y475777H8ozi5/NyI2ast3rt021Ws2sG27ukyGTSoY11GZsU31mP9mSWQ8s6aPHAQCAvtGj8F787HNJkjGbjM4DDz+aJDn6yCMy9atfTJI8tfCZbh3nX7/zg+z7rg/kmJM+lTvvvqfbj1+pVDKqsS7T5zT3ZGwAAOh3PbrVpKb2pU5f+uKLefDhR1OpVLL12C3TOGRIkqTaWV3jMT5x/NHZetyWGVQ3KL+68Zac9rmz86PvfjNbbD7mZftNmzEz02fMTJK0trV1bW+sr8mCJSt6MjYAAPS7Hl3x3nzMpkmSj570qVz7y+tTSTL+ta/JwmcWJUk22rBpjcd403avy9DGxtTXD8rBE/fNjm96Q357x50r7Td50sRccenUXHHp1DTU13dtb2nrzOihg3oyNgAA9LsehfehB09ItVrN4wueSltbW/bY7a0ZMXyD/M//3i6y3eu37fEAlUolqa75SnmSVKvVLGppz6HbNfX4cQAAoD/16FaTww85KCOGb5DZ996XMZtsnMMOOShJMmL4Bjn2w+/PW9+802rXv/DC0tx73/35hx23T21tbX590y35/T335rSTjl3tumqqWdrakUUt7dlxTGMmjB/ek7EBAKDf9Si8k+SAffbKAfvs9bJt+779bdn37W9b49r2jo782/d+mLnzH09NTU3GbbVFvvblz2fcVlusdt2Kjmrqa2syZdfRPscbAIABaY3hfd2sG3p0wIMn7PuKP9uwaUQu//YFPTpekmw7cnAuO2xcj9cBAMC6Yo3h/aVzp750H3Y3VLL68AYAgFerbt1qUu3mmx/TzUAHAIBXmzWG9yUXnFNiDgAAWK+tMbzfvNP2JeYAAID1Wo8/1aS9vT3X/vL63Hn3PXnhhaX51r+clbvvuTfVavL6bbdJY+OQvpgTAAAGtB6Fd0vLspxw2ufy5wceTrVa7XrT5Xcvvzr/8/vZ+eRJx+a9kyf1yaAAADCQ9egDsb/971fmvvsfWunNlu+bPCnVajU3//b2Xh0OAADWFz0K7xtvvi2VSiXnn3X6y7bvvMMbkyRz5z/ee5MBAMB6pEfhvejZZ5Mke/zjW162va6uNknS/PySXhoLAADWLz0K7+EbbJAkeWLBUy/bfuMt//3Sz4dv0EtjAQDA+qVH4f2Xjxb85Oe/0rXtU184J2edf2EqlUresvMOvTsdAACsJ3oU3sd95AMZMrgh8x9/ousTTW7+7e3p6OjIkMEN+ehR7+uTIQEAYKDrUXiP22qLfOeb52WXnbdPpVJ56SMF89KV8G9feG7GbbVFH40JAAADW4//gM74174m//aNc7K8tTUvvLA0G2wwLIMbGvpiNgAAWG/0OLyT5A9/nJPZ987JM4sWZ+NRI7PT9m/Mjm96Q2/PBgAA640ehfcLLyzN5846P3fc9YeVfvaPu+ycr5zxqWywwbDemg0AANYbPbrH+/wLL8ntd96darW60n9uv/PunP/Nb/fVnAAAMKD16Ir3LbfdkUqlkl122j7HfOi92XjUyDyzaHG+f8XV+Z+778ktt/qT8QAAsCo9Cu9BgwZl2fLWnPOFT6epaUSSZOyWm+c147bKhMkfSoM3WQIAwCr16FaTgybskyRZ9OxzL9v+7HPNL/38gH16ZyoAAFjPrPGK93Wzbuj6euuttsxGGzblE5/+Yt590AHZZONRefqZRbn2F7/OqJEbZuyWm/fpsAAAMFCtMby/dO7Urr9S+de+d8XVL/u+Wq3mq9+4KIccPKH3pgMAgPVEt+7xrlar3TpY9/YCAIBXnzWG9yUXnFNiDgAAWK+tMbzfvNP2JeYAAID12lr9yfj7H3wk8x9/Iq1tbSv97OAJ+/7dQwEAwPqmR+Hd/PySnHr6l/KnPz+4yp9XIrwBAGBVehTeF3/38tx73wOvvMMqPv0EAADo4R/Que13d6VSqeS4j3wgSVKpVHLBOV/Ijm96Q7bcfEwu+OoX+mRIAAAY6HoU3ov/9y9WfuDwd3dt22O3t+QrZ3w6jz3xZP7rt7d3+1jzH38iu+9/aM748tfWuO+Di5fn6Gvm5to5zWlt7+zJyAAAsE7oUXjX19cnSRrqG9LQ8NLX8x9/IjU1L91icsN//bbbxzpv6iXZ7vXbdu9xaytp6+jMRXcszBnXLxDfAAAMOD0K75EbbZjkpTdZbrX5ZkmS4085PR858Z+SJHV13btlfNYNN2eDYUPzln/YsZuPXMmwhtqMbarP7CdbMuuBJT0ZGwAA+l2Pwvv1226TarWaOfc/kIn7vyPVajWLFj+Xhc8sSpIcsM+eazzG0hdb8u3LrswpJ360x8NWKpWMaqzL9DnNPV4LAAD9qUefavLpTxyf4z7ygWy0YVP22n3X1NTU5Mabb82KFe3ZZuuxOfyQg9d4jEu+f0XedeAB2XT0xqvdb9qMmZk+Y2aSvOzzwhvra7JgyYqejA0AAP2uR+Hd1DQiTU0jur7/4BGH5oNHHJr//t1d+cRnzswvf31T7rjx2ldcf/+Dj+R3d83Old+5cI2PNXnSxEyeNDFJstcRJ3Ztb2nrzOihg3oyNgAA9Lu1+suVr6S6hp/f9Yc/ZsFTT+fgI45OkrQsW57Ozs48cuzJ3YrxarWaRS3tmbLr6F6YFgAAyunV8F6TyZMm5IB99ur6/odXT8uCp57O6adNWe26aqpZ2tqRRS3t2XFMYyaMH97XowIAQK8qGt6DBw/O4MGDu74fMmRwGurrs+Ff3b6yKis6qqmvrcmUXUdnwvjhaajr0XtCAQCg360xvH8/+941HuShR+at1YMff/SR3dpv25GDc9lh49bqMQAAYF2wxvA+/pTTU6lUSswCAADrrW7dalKtrultkwAAwOqsMbwPnrBviTkAAGC9tsbw/uJnTykwBgAArN98PAgAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAC6ko/4Blf/lp+9/vZWb58eUZutGGOet97csjBE1a75sHFy3P0NXNz6HZNmTB+eBrq/HsBAICBpXh4f+TIw3PGp09Off2gzJ33WI4/5fS8bttt8obXvfYV19TXVtLW0ZmL7liY385fmrP320x8AwAwoBSv1222Hpv6+kEvfVOpJJVKHl/w5BpWVTKsoTZjm+oz+8mWzHpgSZ/PCQAAvan4Fe8kOfeCizNj5g1pbW3N67bdJm/bdZduratUKhnVWJfpc5rzru2a+nZIAADoRf0S3p899cR86hPH549/+nPu/MMf/+8K+F+ZNmNmps+YmSRpbWvr2t5YX5MFS1YUmxUAAHpDv90oXVtbm512eGMWPrMo1/zsFyv9fPKkibni0qm54tKpaaiv79re0taZ0UNXDnUAAFiX9fs7FDs6Ortxj/dLqtVqFrW051C3mQAAMMAUDe9nn2vOrBtuTkvLsnR0dOS/f3dXZt14c3bZecfVrqummqWtHZnX3JYdxzRmwvjhhSYGAIDeUfQe70qlkp9e+8t89RsXp1rtzKabjM4nTzo279jjH1e7bkVHNfW1NZmy62if4w0AwIBUNLw3bBqRSy88t8frth05OJcdNq73BwIAgEJcOgYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFDIjwfnDx8hx9zdxcO6c5re2d/T0OAAD0WF3JB2trW5Fzp16c3931hyxZsjRbbD4mU449Km/bdZfVrquvraStozMX3bEwv52/NGfvt1ka6gbEvxkAACBJ4SveHR0d2WTjUbl06rn5r59fnY8f88GcfuZ5WfDk02tYWcmwhtqMbarP7CdbMuuBJUXmBQCA3lI0vIcMGZzjjz4ym43ZJDU1Ndlz97dmszGb5L4HHurW+kqlklGNdZk+p7lvBwUAgF5W9FaTv7X42ecy/7Enss24rVb62bQZMzN9xswkSWtbW9f2xvqaLFiyotiMAADQG/otvNvb23PGl7+Wgybum3Fjt1zp55MnTczkSROTJHsdcWLX9pa2zoweOqjYnAAA0Bv65R2KnZ2dOeMrX0/doLp85uQTur2uWq1mUUt7Dt2uqe+GAwCAPlD8ine1Ws3Z538zzz7XnAvPOzN1dWseoZpqlrZ2ZFFLe3Yc05gJ44cXmBQAAHpP8fD+6jcuyqPzHsvFX/9yBjc0dGvNio5q6mtrMmXX0ZkwfriPEgQAYMApGt5PPrUw02bMTP2gQZkw+UNd2z/3ySl55/57v+K6bUcOzmWHjSswIQAA9I2i4T1m09G587+uK/mQAACwTnDPBgAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFCG8AACigeHhfPW1GPnTcKdlt/0Ny5lcv6NaaBxcvz9HXzM21c5rT2t7ZxxMCAEDvKx7eG48amY9+6L151zv37/aa+tpK2jo6c9EdC3PG9QvENwAAA07x8N5nr93zjj13y4jhw3uwqpJhDbUZ21Sf2U+2ZNYDS/psPgAA6AsD6h7vSqWSUY11mT6nub9HAQCAHqnr7wFeybQZMzN9xswkSWtbW9f2xvqaLFiyor/GAgCAtbLOhvfkSRMzedLEJMleR5zYtb2lrTOjhw7qr7EAAGCtDKhbTarVaha1tOfQ7Zr6exQAAOiR4le829s70tHRkc7OjnR0dqa1tS21tbWpq6t9xTXVVLO0tSOLWtqz45jGTBjfkzdmAgBA/yse3t+74sf5zg+u6vr+l7++Kcd++P05/ugjX3HNio5q6mtrMmXX0Zkwfnga6gbUhXoAACgf3scffeRqI3tVth05OJcdNq5vBgIAgAJcOgYAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAoQHgDAEABwhsAAAoQ3gAAUIDwBgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAFFA/v55e8kH/6/Jezx8T35OD3Hp2Z1//XGtc8uHh5jr5mbq6d05zW9s6+HxIAAHpZXekHPG/qv2XQoEH51bQf5oGHHsnJp38p226zdbbZeuwrrqmvraStozMX3bEwv52/NGfvt1ka6lysBwBg4Char8uWLc+Nt9yWE475YBobh2SnHd6YvXbfNb/41U1rWFnJsIbajG2qz+wnWzLrgSVF5gUAgN5SNLznPf5EamtqMnbLzbu2jd9m6zwyd1631lcqlYxqrMv0Oc19NCEAAPSNoreaLFu2LMOGNr5s27BhjXmxZdlK+06bMTPTZ8xMkrS2tXVtb6yvyYIlK/p2UAAA6GVFw3vIkCFZ+jeR/eKLLRnaOGSlfSdPmpjJkyYmSfY64sSu7S1tnRk9dFDfDgoAAL2s6K0mY7fYPB0dHZn/+BNd2x54+NG8Ztwrv7Hyr1Wr1Sxqac+h2zX10YQAANA3iob3kCGDs/eeu+WS71+ZZcuW5w9/nJObb70jBx6w92rXVVPN0taOzGtuy45jGjNh/PBCEwMAQO8o/nGCnz31xJx13oXZ/9AjM2L48Jx+6omr/SjBJFnRUU19bU2m7Do6E8YP91GCAAAMOMXDe8TwDfL1r3y+R2u2HTk4lx02rm8GAgCAAlw6BgCAAoQ3AAAUILwBAKAA4Q0AAAUIbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFCC8AQCgAOENAAAF1PX3AN3RNGJ4f4/AOmjs2LH9PQIwQHi9AF7mzOf75WEr1Wq12i+PDAAAryJuNQEAgAKENwAAFCC8AQCgAOENAAAFCG8AAChAeAMAQAHCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoADhDQAABQhvAAAooK6/B+iOIz5yYhrq6/t7DNYxzz3/fDYcMaK/x2Ad47xgVZwXrIrzgr/V2taW//j3i/vs+AMivBvq63PFpVP7ewzWMR867hTnBStxXrAqzgtWxXnB3/rQcaf06fHdagIAAAUIbwAAKGBAhPehkyb29wisg5wXrIrzglVxXrAqzgv+Vl+fE5VqtVrt00cAAAAGxhVvAAAY6IQ3AAAUsE58nODzS17I2edfmNvvvDtNI4bnpGM/nIn7vWOV+175k//M5T+6Jsvb2rLPXrvn9FOnpL5+UNmB6RW99Xtf3XEWPPl03vX+j2bI4MFdx/rwB96Tjx31/r5+evSS3jpPrp42I9fNvCEPPTo3E/Z5e848/dSCz4K/V4nzwOvFwNcb50lb24qcO/Xi/O6uP2TJkqXZYvMxmXLsUXnbrruUfTKstRLnwVq/XlTXAad/6bzqZ888t/riiy3Vu2ffW93rwMOrDz0yd6X9brvjzur+hxxZfeiRudXnl7xQPfYTn6l+85LLyg9Mr+it3/vqjvPEgqeqb377QdUVK9pLPS16WW+dJzfcfGv1pltuq57z9X+tfvGcbxR8BvSGEueB14uBrzfOk5aWZdVLvv/D6hMLnqp2dHRUb7n1juqeEw+rPrHgqcLPhrVV4jxY29eLfr/VZNmy5bnxlttywjEfTGPjkOy0wxuz1+675he/ummlfa+bdWPefeD+2WbrsRm+wbB87Kj35bqZ1/fD1Py9euv33pPjMPD05uvDPnvtnnfsuVtGDB9e8inQC5wHdEdvnSdDhgzO8Ucfmc3GbJKamprsuftbs9mYTXLfAw+VfkqshXX9POj38J73+BOpranJ2C0379o2fput88jceSvt+8jcedl2m61ftt/i55rT/PySIrPSe3rr997d40x639E58LAP50vnTk1z8/N98IzoC14fSMqfB14vBqa+Ok8WP/tc5j/2RLYZt1XfDE6vKn0e9PT1ot/De9myZRk2tPFl24YNa8yLLctW2rdl2fIMGzr0r/Z76euWVezLuq23fu9rOk7TiOG5/JILMuPHl+WKS6fmxZaWfP4rX+vtp0Mf8fpAUu488HoxsPXFedLe3p4zvvy1HDRx34wbu2UfTE1vK3UerO3rRb+H95AhQ7L0b57giy+2ZGjjkJX2bRwyOC+2tHR9v/TFl75uXMW+rNt66/e+puM0Ng7Jdq/fNnV1tRm50Yb59Mkfz+3/c3fXMVi3eX0gKXceeL0Y2Hr7POns7MwZX/l66gbV5TMnn9BHU9PbSp0Ha/t60e/hPXaLzdPR0ZH5jz/Rte2Bhx/Na8aNXWnf14wbmwcefrTr+wcffjQjN2xK0wj36g00vfV778lxkqRS+d8v/N2oAcHrA0n/nQdeLwaW3jxPqtVqzj7/m3n2ueacf9bnUle3TnwIHN3QX+dBd18v+j28hwwZnL333C2XfP/KLFu2PH/445zcfOsdOfCAvVfa96AD9sm1P/9VHpk7P0teWJrvXXF1Dp64Xz9Mzd+rt37vazrOvXPuz9z5j6ezszPNzy/J1751ad680/Zd/3cS67befH1ob+9Ia2tbOjs70tHZmdbWtrS3d5R8OqylUueB14uBrTfPk69+46I8Ou+xXHDOFzK4oaHk0+DvVOo8WNvXi3XiT8Y/v+SFnHXehbnjrrszYvjw/L/jXvq8xaeeXpjDP3xifvKDi7PpJqOTJD/8j+m5/KqfprW1Nfvs9bacfprP8R6oeuv3/krHSZKZN9yci79zeZ5tbs7QxsbsustO+cTxx2TUyA3762nTQ711nnz7sivznR9c9bJjH/vh9+f4o48s/pzouRLngdeLga83zpMnn1qYSe87JvWDBqW2trbr2J/75JS8c/+V4411T4nzYG1fL9aJ8AYAgPVdv99qAgAArwbCGwAAChDeAABQgPAGAIAChDcAABQgvAEAoAB/igmgF0167zF58umFa9zvkgvOyZNPLcyXzpva9f0uO+/Qx9Ot3oInn8673v/Rl207bcqx+cDh7+7W+h/95Gf5xkXfedm2a6/6XjYbs0mvzQgwkLniDQAABfgDOgB95M6778kJp34uSXLwhH1z5umn9vNEq/fXV7y/+JlTMumd+61hxarN+OX1XVfyXfEG+D9uNQHoJ38dqH+51eSvY/0zp3w8f37g4fz6pt9kgw2G5YSjj8yBB+ydb1/2o0yb8cvU1tbmwP33zpRjP5y6uv/7k8b3zrk/3/vh1bnn3vvyYsuybLbp6Bx4wN75yAcOT13d2r3sd3R05LIr/yMzr785Ty18JjWVmmw8amS2e/22+cTxH8nGo0b+3f99AKzvhDfAOuqS7/8wzy95IUnSsmxZzjr/wtz4m9vym9t+17XPFVdPy+abbZrD3n1gkuS/f3dXTv3c2Wlvb+/aZ/7jC3LJ96/Mn+57IBd89YtrNcsVV0/PJd+/8mXb5j32eOY99nje/553CW+AbnCPN8A6anBDQ6b/8NJccM4XkiTVajW33n5nvnX+l3LtVd9L45AhSZIbb761a815Uy9Je3t7dnjTGzLjx9/PrbOm5bQpxyZJfvPf/5Pb7rhrrWaZ/cc/JUl2eNMbctOMH+eWX/wkV33vWznpuI9k+PAN/p6nCfCq4Yo3wDpq0jv3y5ZbbJbRG4/q2rbjm96Q3d765iTJa18zNvf86c956plFSZJ5jz2Rxxc8mSS55977Mul9x6x0zDv/cE923/XNPZ5l001GJ0kenTs/37n8qrz2NeMy/rWvyYff/55UKpUeHw/g1Uh4A6yjNh29cZKkoaF+pW1Juu7XXtG2IknyXPPzazzmkv+9daWnPnbU+/Lgw4/mD3+ckx/95Gdd27facvP86/lneQMlQDcIb4B1VG1tbbe2/UXTiOFdX7//sHfnkycdu9I+a/tBViM32jDf/db5WfjMojz0yLw89OjcfPcHP878x57I9394dT7/qU+s1XEBXk3c4w2wnhi75ebZ/H+vPP/s57/KrXfcmdbWtjzX/Hxm3XBzjjrh1Dz51Jr/uM+qTJsxM7/89U1Z0d6eXXbeIQfsvWeGDx+WJHmueUmvPQeA9Zkr3gDriUqlkk+dfEI++c9fTsuyZTn5M2f22rHvufe+XDfrhlX+bLe3/kOvPQ7A+kx4A6xH9vjHt+S73zwvl/3oJ5n9x/vSsqwlG224YbbZeqvsvedu2XjURmt13H3evnuWLF2a+x98JM3Nz2fQoEHZfLNN8+4D9+/6KEMAVs9frgQgib9cCdDX3OMNwEq+dN7U7PKOg1/2CSZr8qOf/Cy7vOPgrugG4OWENwAAFOBWEwAAKMAVbwAAKEB4AwBAAcIbAAAKEN4AAFCA8AYAgAKENwAAFPD/AQCzJ9Lal/VGAAAAAElFTkSuQmCC"
     },
     "metadata": {}
    }
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [],
   "outputs": [],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "language": "python",
   "display_name": "Python 3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}